Skip to content

Conversation

@Wavezy11
Copy link

Title: feat(sinks/postgres): Add columns configuration option

Summary

Add support for explicit column specification in PostgreSQL sink to address issue with default values and serial columns. This allows users to exclude columns like serial/auto-increment fields that should be handled by PostgreSQL itself.

Changes:

  • Add optional columns parameter to PostgresConfig
  • Modify PostgresService to use specified columns in SQL queries
  • Update documentation warnings to mention new feature
  • Add test coverage for columns configuration

Vector configuration

sinks:
  my_sink_id:
    type: postgres
    inputs:
      - my-source-or-transform-id
    endpoint: postgres://user:password@localhost/default
    table: table1
    columns:
      - column1
      - column2

How did you test this PR?

  • Added unit tests to verify columns configuration parsing
  • Tested SQL query generation with and without columns specified
  • Verified backward compatibility when columns parameter is empty
  • Manually validated configuration serialization/deserialization

Change Type

  • Bug fix
  • New feature
  • Non-functional (chore, refactoring, docs)
  • Performance

Is this a breaking change?

  • Yes
  • No

Does this PR include user facing changes?

  • Yes. Please add a changelog fragment based on our guidelines.
  • No. A maintainer will apply the no-changelog label to this PR.

References

Notes

  • This PR addresses the limitation where PostgreSQL's default values and serial columns are not supported due to the jsonb_populate_recordset function
  • The new columns parameter allows users to explicitly specify which columns to insert, excluding those that should be handled by PostgreSQL's default mechanisms
  • Backward compatible - when no columns are specified, the original behavior is preserved

Add support for explicit column specification in PostgreSQL sink to address
issue with default values and serial columns. This allows users to exclude
columns like serial/auto-increment fields that should be handled by PostgreSQL
itself.

Changes:
- Add optional `columns` parameter to PostgresConfig
- Modify PostgresService to use specified columns in SQL queries
- Update documentation warnings to mention new feature
- Add test coverage for columns configuration
@Wavezy11 Wavezy11 requested review from a team as code owners January 29, 2026 13:04
@github-actions github-actions bot added domain: sinks Anything related to the Vector's sinks domain: external docs Anything related to Vector's external, public documentation labels Jan 29, 2026
@github-actions
Copy link


Thank you for your submission, we really appreciate it. Like many open-source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution. You can sign the CLA by just posting a Pull Request Comment same as the below format.


I have read the CLA Document and I hereby sign the CLA


You can retrigger this bot by commenting recheck in this Pull Request. Posted by the CLA Assistant Lite bot.

@Wavezy11
Copy link
Author

@cla-assistant recheck

@Wavezy11
Copy link
Author

I have read the CLA Document and I hereby sign the CLA

recheck

@Wavezy11
Copy link
Author

https://github.com/cla-assistant recheck

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

domain: external docs Anything related to Vector's external, public documentation domain: sinks Anything related to the Vector's sinks

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(postgres sink): Allow ignoring serial columns in target table

2 participants