fix: load paths, imports, transpile, and CI permissions

[Chessing234]

Summary

• Quote/escape data dirs and passwords in postgres/sqlite/duckdb loaders
• Accept plain .csv (and .csv.gz) in duckdb/sqlite imports; utf-8 transpile writes
• sqlfluff annotate no longer fails fork PRs on missing permissions
• Concept SQL kept out (see Fix SOFA CV scoring, urine_output_rate diffs, and Elixhauser no_drg #2126 and follow-ups)

Test plan

• pytest tests/test_transpile.py

Made with Cursor

[Chessing234]

Infra-only follow-up to the too-large #2124. Concept changes stay in #2126 / separate PRs.

[alistairewj]

the changes are scattered and not very cohesive but with some cleaning up we can merge

[Chessing234]

Addressed review feedback in 270b6d3:

• SQLFluff: removed continue-on-error; added permissions: contents: read + checks: write; use ignore-unauthorized-error: true
• Reverted duckdb usage text (no .tar.gz), DuckDB GENERATE_ARRAY transform + related concept/test, Makefile concepts target, CheXpert/NegBio defensive rewrites, and validate_concepts awk change
• Removed sqlite TABLE_DTYPES from this PR (will split to a follow-up with CI as requested); kept plain .csv loading

[Chessing234]

split the review asks into focused follow-ups: sqlfluff annotate permissions are in #2148; the sqlite import.py csv/.csv.gz discovery (with unit tests + ci) is in #2149. happy to trim the remaining infra bits on this branch further if you want them dropped.

[acnimma]

@alistairewj Ready for another look.

Your July 25 review asks are addressed as follows:

Ask	Status
SQLFluff continue-on-error too blunt	Removed; permissions + ignore-unauthorized-error moved to #2148
DuckDB .tar.gz usage text	Reverted
DuckDB GENERATE_ARRAY transform	Reverted
Makefile concepts target	Reverted
CheXpert / NegBio defensive path edits	Reverted
sqlite CSV discovery	Split to #2149 with tests/CI
validate_concepts.sh awk / comment	Reverted

This PR now keeps only the remaining load-path / transpile / duckdb import / validate_demo infra. Happy to cut anything else that still feels out of scope.

[Chessing234]

reverted the make-concepts.sh quote-only hunks too. should be down to duckdb path escaping, postgres datadir/user quoting, sofa/oasis build order, transpile utf-8, and validate_demo tables. sqlfluff + sqlite csv bits stay in #2148 / #2149.

[Chessing234]

@alistairewj resolved the review threads — ready for another look when you have time.

[Chessing234]

@alistairewj re-review please when you get a chance — pushed after your review to address each point:

• dropped the defensive CheXpert/NegBio path rewrites and the validate_concepts.sh awk change (reverted, 3d189fe1 / f700337f)
• reverted the duckdb GENERATE_ARRAY/UNNEST dialect change — pinned sqlglot already handles it
• reverted the broken postgres Makefile concepts target path
• SQLFluff continue-on-error + workspace permission moved to focused follow-up ci: fix SQLFluff annotation permissions for fork PRs #2148 (ignore-unauthorized-error: true for fork PRs)
• the sqlite plain .csv/.csv.gz import split out to fix(mimic-iii): load plain CSV in sqlite import.py (prefer .csv.gz) #2149 with unit tests + CI, as you asked

kept this PR scoped to load-path / transpile / import infra only.