Possible read-after-write consistency issue with multiple schema migration steps in Iceberg tables on AWS Glue + S3 #2599
Description
Apache Iceberg version
Pyiceberg 0.10.0
Pyiceberg-core 0.6.0
Please describe the bug 🐞
This may be a hard one to pin down but I noticed that multiple schema migration steps executed sequentially in the same update_schema context sometimes result in Exceptions like column name not found when using Iceberg tables on AWS Glue. An example:
with table.update_schema() as update:
update.rename_column("some_column", "renamed_column")
update.move_first("renamed_column") # this sometimes fails with an error
# that renamed column doesn't existI have not noticed it with other back-ends like SQLite, leading me to believe it is a Glue issue specifically where a write may not yet be reflected by the time of the next operation.
Willingness to contribute
- I can contribute a fix for this bug independently
- I would be willing to contribute a fix for this bug with guidance from the Iceberg community
- I cannot contribute a fix for this bug at this time
Activity
jayceslesar commentedon Oct 10, 2025
hmmm we should definitely see if we can reproduce in CI via adding to #2371 (which was just merged)
Ideally we also set up integration tests for glue there as well
github-actions commentedon Apr 9, 2026
This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs. To permanently prevent this issue from being considered stale, add the label 'not-stale', but commenting on the issue is preferred when possible.
Resolve staged renames when moving columns in UpdateSchema
David-Banquet commentedon Oct 4, 2026
I tried to reproduce this on current
main(068aae5) with a table that hasidandsome_column:move_first("renamed_column")move_first("some_column")ValueError: Cannot move missing column: renamed_columnrenamed_column, idValueErrorIt fails every time for me, on both catalogs, so I don't think Glue consistency is involved. Inside one
update_schema(), themove_*methods look names up in the schema as it was when the update started, plus columns added in the same update. Pending renames are not considered (_find_for_move, same code in 0.10.0). If you pass the original name, the move works and the rename is still applied.The Java implementation behaves the same way.
SchemaUpdate.findForMovechecks columns added in the update, then callsschema.findField(name)on the base schema (SchemaUpdate.java#L418-L430). Names in the Java API refer to the base schema in general: a Java test renamespreferences.feature2in the same update that renamespreferences, still using the old parent name (TestSchemaUpdate.java#L440-L441).#3687 proposed resolving pending renames in
_find_for_move. That would make PyIceberg accept a call that Java rejects, so I'd keep the current behavior and make it easier to understand instead: when the name only matches a pending rename, the error would say to use the original name, and the "Rename column" section of the docs would mention it. I can open a small PR for that if maintainers agree.@din14970, the part I can't explain is the "sometimes". Do you remember what changed between the runs that passed and the ones that failed? For example the PyIceberg version, the code path, or whether the rename and the move always ran in the same
update_schema()block.din14970 commentedon Oct 4, 2026
@David-Banquet unfortunately this was too long ago and I don't remember. It is possible/probable I was simply not diligent enough in trying to figure out the real cause, and maybe changed something else unwittingly between runs which made me come to the wrong conclusion. Your analysis looks plausible. I don't understand why I wrote that it worked for me with SQLite. Would have to dig it up again.