feat: catch server-default, CHECK-constraint and enum drift the diff test misses

[AlexeyShalaev]

Summary

test_migrations_up_to_date is only as thorough as compare_metadata(), and compare_metadata()
does not look at three things: server defaults (unless compare_server_default is in the context
opts), CHECK constraints, and enum members. A suite that trusts "the diff is empty" is trusting it
for a schema that can still have is_active DEFAULT false where the model says true, no
chk_orders_amount_positive at all, and an order_status type the application writes 'shipped'
into.

Three changes, one per gap.

Server defaults — migration_diff_compare_server_default: ClassVar[bool] = False on
MigrationConsistencyMixin, passed straight through as compare_server_default in the
MigrationContext opts. Off by default, so no existing suite changes its result. I measured which
spellings agree on PostgreSQL 17 rather than guessing: Alembic's PostgreSQL comparison compares the
text first and, when it differs, asks the server whether the two expressions are equal, so
text("true"), sa.true() and "true" all agree with a column defaulted to true, func.now()
agrees with now() and CURRENT_TIMESTAMP, "0" with 0, text("'{}'") with '{}'::jsonb. What
it reports is a different value, two different volatile functions (clock_timestamp() against
now()), and a default present on one side only — including a model with a Python-side default=
against a migration with a server_default. That last one is why it stays opt-in; the table in
docs/guide/configuration.md says all of this.

CHECK constraints — test_check_constraints_match, comparing by name, per table, in both
directions. I did not re-derive names from the convention: SQLAlchemy already does that when the
constraint attaches to the table, so CheckConstraint("amount > 0", name="amount_positive") under a
chk_%(table_name)s_%(constraint_name)s convention already carries chk_orders_amount_positive by
the time we see the metadata, and rules_from_metadata only knows prefixes and suffixes, so it could
never produce a name anyway. The two cases that leaves are deferred names
(Boolean(create_constraint=True), resolved at DDL time) and names over 63 characters, which
PostgreSQL truncates with a hash. Alembic's _get_constraint_final_name handles both and is what
autogenerate already uses for index and unique-constraint names, so the check test resolves names
exactly the way the diff test does. A named constraint the models declare and the database lacks is
always reported; one the database has and the models do not is reported except on a table where the
model has an unnamed check constraint, because that one carries whatever name PostgreSQL gave it and
the "unexpected" side would be a false positive.

Enum members — test_enum_values_match, every native, named Enum column, its labels read
through the inspector's get_enums() (ordered by enumsortorder) and compared as a list against
Enum.enums, so a value added in the wrong position is reported too. A type in the database that no
column uses is not this test's business.

Both tests go on MigrationConsistencyMixin, not a fourth mixin: they ask the same question as
test_migrations_up_to_date with the same four fixtures, MigrationTestBase would include a new
mixin anyway, and a separate one would only add a name to learn. The five existing tests keep their
names and behaviour, and migration_diff_ignore_tables applies to both new tests. The comparisons
themselves live in utils/diff.py as two sync helpers next to is_ignored_diff_item, so they are
unit-testable and usable from a hand-written check.

Rejected along the way: comparing CHECK constraints by SQL text (PostgreSQL rewrites amount > 0 as
(amount > (0)::numeric), so it would only ever match by luck); reading pg_enum by hand, as the
issue's lab does, when the inspector already returns the labels in order; and turning the
server-default comparison on by default.

One dependency note. The lock moves alembic 1.19.1 → 1.19.2 (3 lines, alembic only), and it is
load-bearing here rather than housekeeping: 1.19.0 shipped name-based CHECK-constraint detection as
an autogenerate plugin that was on by default, and 1.19.2 renamed it to
alembic.ext.checkconstraint_byname and took it out of the default plugin set. On 1.19.0 and 1.19.1
the diff test therefore reports named CHECK constraints as well, which would make
TestDriftCheckConstraintMissing fail on a test it is not about. Pinning CI to the version users
install today keeps the new test's subject unambiguous. Nothing in the change depends on it: both
helpers work on the declared floor, verified below.

Type of change

• Bug fix
• New feature
• Breaking change
• Documentation update
• Refactoring / internal

Checklist

• Tests added or updated — tests/integration/test_drift_issue_37.py (a new sample app in four
  variants: clean, wrong server default, missing check, enum without shipped) and 15 unit tests for
  the two helpers in tests/unit/test_diff.py
• make check passes locally (ruff + mypy)
• CHANGELOG.md updated under [Unreleased] — n/a, Release Please owns it
• Documentation updated — docs/guide/quickstart.md (what each test catches, two new failure
  modes in Troubleshooting), docs/guide/configuration.md (the new attribute and the table of
  spellings that agree), docs/guide/advanced.md, docs/index.md, docs/reference/index.md,
  README.md, and docs/agents.md (the seven tests, the two helpers, three new rules)

Verification

The full gate, on alembic 1.19.2:

$ make check
uv run ruff check .        All checks passed!
uv run ruff format --check .   47 files already formatted
uv run mypy alembic_gauntlet   Success: no issues found in 17 source files

$ make test
139 passed, 11 warnings in 6.66s
Required test coverage of 63% reached. Total coverage: 67.09%

The negative control — the same new tests with alembic_gauntlet/ restored to master's version:

$ git checkout HEAD~1 -- alembic_gauntlet/ && pytest tests/integration/test_drift_issue_37.py
FAILED TestDriftServerDefaultCompared::test_migrations_up_to_date
    Failed: DID NOT RAISE AssertionError
FAILED TestDriftCheckConstraintMissing::test_check_constraints_match
    AttributeError: 'super' object has no attribute 'test_check_constraints_match'
FAILED TestDriftEnumValueMissing::test_enum_values_match
    AttributeError: 'super' object has no attribute 'test_enum_values_match'
3 failed, 14 passed

The lab from the issue, tests/test_gauntlet.py — the inherited suite, nothing hand-written —
against a wheel built from this branch:

VARIANT=clean                   7 passed
VARIANT=drift_server_default    7 passed            (attribute off: not compared, as documented)
VARIANT=drift_server_default    1 failed, 6 passed  (attribute on)
  E  AssertionError: Database schema is out of sync with ORM models. Differences:
  E    [[('modify_default', None, 'users', 'is_active', {...}, DefaultClause(...), DefaultClause(...))]]
VARIANT=drift_check_missing     3 failed, 4 passed
  E  AssertionError: CHECK constraints are out of sync with ORM models:
  E    Check constraint 'chk_orders_amount_positive' on table 'orders' is in the models but not in the database.
VARIANT=drift_enum_value        1 failed, 6 passed
  E  AssertionError: Enum values are out of sync with ORM models:
  E    Enum type 'order_status' has values ['new', 'paid'] in the database and ['new', 'paid', 'shipped'] in the models.

drift_check_missing fails three tests rather than one because that variant's revision 0003 creates
nothing on upgrade but still drops the constraint on downgrade, so the stairway and full-downgrade
tests hit constraint "chk_orders_amount_positive" of relation "orders" does not exist. The new test
is the one that names the drift itself.

Floor check, because _get_constraint_final_name and PGInspector.get_enums are the two new
imports: on a fresh venv with alembic==1.8.0 and sqlalchemy==2.0.0, the declared minimums, both
helpers import and return the same lines as on 1.19.2 — including the deferred
Boolean(create_constraint=True) name, resolved to ck_users_active on both.

Closes #37

[codecov]

Codecov Report

❌ Patch coverage is 74.35897% with 10 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
alembic_gauntlet/utils/diff.py	74.35%	10 Missing ⚠️

📢 Thoughts on this report? Let us know!