Repository navigation
sql: add CREATE SINK ... INTO POSTGRES - #39704
patrickwwbutler wants to merge 1 commit into
Conversation
Parse, purify and plan Postgres sinks, add the PostgresSinkConnection storage type, and teach the catalog and mz-deploy about the new sink connection. The storage dataflow is left as an unimplemented stub. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
peterdukelarsen
left a comment
There was a problem hiding this comment.
Def would pref a feature flag earlier gating the SQL to make it easier to merge incrementally.
| /// on the conservative side and return `true`. | ||
| fn redact_value(&self) -> bool { | ||
| match self { | ||
| PostgresSinkConfigOptionName::Schema | PostgresSinkConfigOptionName::Table => false, |
There was a problem hiding this comment.
Do we have docs on what's safe/unsafe to log? I assume table names are not considered sensitive even though they're user-entered?
| CreateSinkConnection::Postgres { connection, .. } => { | ||
| // Only check that the referenced connection is a Postgres one. | ||
| // TODO: reach the upstream server here, the way the Kafka arm above | ||
| // proves the broker is reachable. |
There was a problem hiding this comment.
Not-blocking more something to think about medium term: Moritz has described to me a vision where we avoid calls to upstream systems in purification and instead just report errors to connect/validate vie the error stream for the created operation. Ideally we'd be able to set up some sort of syntactic sugar so we can still select a success/failure of connection from the sync (otherwise I think not getting immediate feedback is too big of a regression in behavior to be worth the cleanliness of dropping network calls from purification).
| /// The schema holding the target table. | ||
| pub schema: String, | ||
| /// The table to write into. | ||
| pub table: String, |
There was a problem hiding this comment.
any comment we should make about escaped/not escaped?
QA LLM Review1. HIGH --
|
Parse, purify and plan Postgres sinks, add the PostgresSinkConnection
storage type, and teach the catalog and mz-deploy about the new sink
connection. The storage dataflow is left as an unimplemented stub.
Tests: adds parser cases for
CREATE SINK ... INTO POSTGREStosrc/sql-parser/tests/testdata/ddl.NOTE: no feature flag gates Postgres sinks, and until the rest of the stack lands
render_sinkisunimplemented!(), so creating one would panic clusterd. Merge this stack together, or gate the sink behind a flag first.🤖 Generated with Claude Code