Repository navigation
perf: harden runtime guards, expand mypyc surface, and optimize PGO builds - #921
Merged
Merged
Conversation
…ment wire flags, and adapter edge cases
…ult materialization
…s-adapter hot paths
…direct submodule imports
…erage, and fix CI wheel matrix
…ncpg, duckdb, and spanner drivers
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #921 +/- ##
==========================================
- Coverage 84.66% 84.63% -0.04%
==========================================
Files 628 629 +1
Lines 140808 142408 +1600
Branches 30685 30807 +122
==========================================
+ Hits 119219 120529 +1310
- Misses 14999 15244 +245
- Partials 6590 6635 +45
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
…t in compiled builds Compiled driver classes store dialect on the instance, so reading it from the class returned a descriptor: migrations ran without a lock and a deferred request session raised on dialect before its connection opened. The lock backend now reads the instance value for driver adapters, and the deferred session falls back to the configuration's statement config. The PostgreSQL tracking-table guard now rejects row-level security only when it applies to the migrating role.
…cancelled statements In compiled builds, an async with block whose exit suspends left the task pending forever when the body failed through a future or was cancelled. The affected sites in the asyncpg, psycopg, Db2, Oracle, Spanner, mysql-connector, aiomysql, and asyncmy drivers now enter the context directly and exit it outside any exception handler. Also: - psycopg execute_stack accepts list parameters in compiled builds - Cockroach async retries keep the driver error as the cause when the rollback also fails - asyncpg, Cockroach asyncpg, and Db2 mapped exceptions set their cause explicitly - asyncpg releases a failed transaction claim on pooled connections - MySQL drivers accept tuple fetch results, and aiomysql maps only client-raised TypeErrors - psycopg skips percent escaping work for SQL without pyformat placeholders and reads the cursor description once
…and DuckDB overwrite loads - aiosqlite load_from_arrow and load_from_storage raise the mapped error instead of hanging in compiled builds - the SQLite and aiosqlite rowid cache is keyed by table and bounded, so sqlcommenter comments no longer cause a probe per insert - DuckDB load_from_arrow(overwrite=True) falls back to a positional insert when column names differ - aiosqlite runs statements and begin() with one worker round trip instead of three - Teradata rejects a boolean batch size in compiled builds
…estore the validator cache - the cached cursor path converts tuple fetchall() results before row collection - base driver errors set their cause explicitly, which compiled builds otherwise drop - cached statements check observability live, so hooks registered on an open session fire - building a statement cache holder no longer disables the shared parameter validator cache
…through world-writable ancestors - pyproject.toml and project-root discovery no longer stop at a world-writable ancestor; trust is decided by marker ownership, and permission checks are skipped on platforms without POSIX ownership - discovery logs why it stopped or refused a marker - s3a, abfs, and abfss URIs resolve object keys without the bucket prefix - fsspec copy, move, and delete treat glob characters in keys literally and fall back to rm() when rm_file() is not implemented
…time and name invalid settings - session and connection lifecycle hooks registered after the runtime is built now fire - Starlette, FastAPI, Flask, and Sanic validate routing settings and normalize status sets and flags before building compiled state, so invalid values raise the named configuration error - verify_primary_pin returns a failed verification for a missing or non-string token in compiled builds - the Prometheus observer caches label children - the Litestar ambiguous config type error points at the configuration instance
…he wheel smoke test - the smoke script checks the migration lock backend, deferred session dialect, failing and cancelled loads and stacks, psycopg stack parameters, and tuple fetch results, each bounded by a timeout - Linux wheel tests install a hashed driver SDK profile and fail when a guaranteed driver module is skipped - profile training fails on a skipped workload in strict mode and always fails on import errors raised by sqlspec itself
…storage key, and tracking-table changes
…t path and handle list-valued settings - a sticky world-writable directory such as /tmp is never used as the migration project root, even when it holds a trusted marker - recursive fsspec deletes of a literal key work on filesystems whose find() returns a list for a single file - psycopg detects upper-case %S placeholders before skipping percent escaping - framework status overrides given as a list or tuple are stored as a set
…nts without comments Statements whose text ends with a comment are looked up by table only, so per-request sqlcommenter attributes neither probe per insert nor add cache entries, and every other insert keeps the single dictionary lookup.
… when hooks are registered The cached-statement path reads a per-driver flag again instead of evaluating the runtime's idle state on every statement. The runtime counts hook registrations, and a driver recomputes its flag when the count changes, so hooks registered on an open session still fire.
…fore reading it Discovery opens the file once, reads ownership and permissions from the open descriptor, and parses from the same handle, so a file swapped in after discovery is refused instead of read.
…t debug level Running migrations as a different user from the one that owns the source tree, as a container does with a mounted checkout, printed a warning for every migration directory. The search already ended there silently before refusals were logged; a refused marker still warns.
A compiled module does not apply the from clause of a raise statement, so the error raised from a compiled wheel had no __cause__ and showed the handled error as unrelated context. The sites that build the error inline now attach the cause with with_cause(), and the sites that raise a local error set __cause__ before raising, as the database error mappers already do.
…in, commit, and rollback sqlite3 tracks the flag itself and rejects the assignment, so every transaction boundary raised and discarded an AttributeError.
The create-only checks wrote fixed keys, so they failed whenever the bucket outlived the session that first wrote them.
…anager Leaving the cursor context built and entered a contextlib.suppress object for every statement. A plain try/except does the same cleanup for about 280 ns less per statement.
…of five runs A single timing of each size failed on a loaded runner.
…heel smoke run The wheel-test jobs now install the BigQuery, Spanner, mssql-python, SQL Anywhere and Db2 SDKs so their driver modules are imported from the built wheel instead of skipped. Db2 is installed on x86_64 only, where ibm_db publishes a wheel, and sqlanydb is built from its hash-pinned source distribution with a hash-pinned setuptools. The smoke script constructs every adapter driver class around a placeholder connection, accepts --require-sdk for SDKs required on a single job, and checks that an error raised while handling another keeps its direct cause or its suppression.
Recaptured interpreted at 1000 rows, pool size 1, seven iterations and three warmups on Python 3.11. The SQLite baseline gains the raw and SQLSpec declared_parameters rows and the subsystem baseline gains the two declared-parameter check probes.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Compiles 39 more modules with mypyc (234 to 273, including every adapter driver), speeds up the statement and driver hot paths, and tightens migration, config-discovery and storage handling. It also fixes the behaviour differences that compiling the drivers exposed, so a compiled wheel behaves the same as a source install.
What changed
raise ... from ...lost the original error. Raise sites in compiled modules now attach the cause themselves, through a smallwith_cause()helper insqlspec.exceptions.FSSpecBackendtreats*,?and[in keys literally on copy, move and delete.aiomysqlandasyncmyturn off multi-statement execution on each connection.How you use it
Most of this needs no action. A few behaviours change:
aiomysql,asyncmy):execute()now rejects stacked statements. Useexecute_script()for multi-statement SQL, or opt back in withconnection_config={"client_flag": CLIENT.MULTI_STATEMENTS}.get_annotation,provide_db_sessionandprovide_db_connectionraiseKeyErrorwhen a config class is shared by several registered configurations. Pass the configuration instance or its session, connection or pool key.commit_mode="manual", and an explicit autocommit mode raisesImproperConfigurationErrorat plugin creation. Callcommit()in the handler to keep work done inside an explicit transaction.FSSpecBackendexpanding glob characters in copy, move or delete must list the matching keys first.Docs
docs/changelog.mddescribes the behaviour changes above.docs/databases/mysql.mddocuments the multi-statement default foraiomysqlandasyncmy.docs/learn/migrations.mdupdates the tracking-table and schema-extraction rules.