Skip to content

feat: implement bulk deletion for tables with composite primary keys - #1896

Merged
Artuomka merged 2 commits into
mainfrom
backend_bulk_delete_optimisation
Oct 14, 2026
Merged

Artuomka merged 2 commits into
mainfrom
backend_bulk_delete_optimisation

Conversation

@Artuomka

@Artuomka Artuomka commented Oct 14, 2026 •

Copy link
Copy Markdown
Collaborator
  • Added end-to-end tests for bulk deletion of rows in PostgreSQL, handling cases where the number of keys exceeds statement limits and ensuring proper logging of deletions.
  • Introduced unit tests for primary key normalization and filtering, ensuring correct handling of composite keys and null values.
  • Refactored data access objects for ClickHouse, MSSQL, MySQL, Oracle, and PostgreSQL to utilize new bulk primary key handling logic, improving efficiency and correctness in bulk delete operations.
  • Created helper functions for normalizing primary keys, chunking them according to database limits, and applying filters in queries.
  • Updated error messages for clarity regarding primary key requirements.

Summary by CodeRabbit

  • Bug Fixes
    • Bulk row reads and deletions now handle large requests in bounded batches across supported databases, improving reliability for operations involving many rows.
    • Primary-key values such as 0, false, and null are preserved during row lookup, updates, and deletion. Invalid or inconsistent primary-key fields are rejected.
    • Bulk deletion results now report the number of rows actually deleted, and matching row data is included in deletion results when available.
  • Tests
    • Added coverage for large bulk deletions, composite keys, null values, and invalid keys.

Artuomka and others added 2 commits October 14, 2026 11:48
- Added end-to-end tests for bulk deletion of rows in PostgreSQL, handling cases where the number of keys exceeds statement limits and ensuring proper logging of deletions.
- Introduced unit tests for primary key normalization and filtering, ensuring correct handling of composite keys and null values.
- Refactored data access objects for ClickHouse, MSSQL, MySQL, Oracle, and PostgreSQL to utilize new bulk primary key handling logic, improving efficiency and correctness in bulk delete operations.
- Created helper functions for normalizing primary keys, chunking them according to database limits, and applying filters in queries.
- Updated error messages for clarity regarding primary key requirements.
@Artuomka
Artuomka enabled auto-merge October 14, 2026 11:48
@coderabbitai

coderabbitai Bot commented Oct 14, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

Note

Currently processing new changes in this PR. This may take a few minutes, please wait...

⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 36265d70-a0f8-47ec-bc0c-f3298298e486
📥 Commits

Reviewing files that changed from the base of the PR and between c80c597 and 38a638d.

📒 Files selected for processing: 22
  • backend/src/entities/table-logs/table-logs.service.ts
  • backend/src/entities/table/table-pure-crud-operations/use-cases/pure-delete-row-from-table.use.case.ts
  • backend/src/entities/table/table-pure-crud-operations/use-cases/pure-read-row-from-table.use.case.ts
  • backend/src/entities/table/table-pure-crud-operations/use-cases/pure-update-row-in-table.use.case.ts
  • backend/src/entities/table/table.controller.ts
  • backend/src/entities/table/use-cases/delete-row-from-table.use.case.ts
  • backend/src/entities/table/use-cases/delete-rows-from-table.use.case.ts
  • backend/src/entities/table/use-cases/get-row-by-primary-key.use.case.ts
  • backend/src/entities/table/use-cases/update-row-in-table.use.case.ts
  • backend/src/entities/table/utils/add-display-names-for-tables.util.ts
  • backend/src/entities/table/utils/build-table-settings-for-response.util.ts
  • backend/src/entities/table/utils/find-objects-with-properties.ts
  • backend/test/ava-tests/non-saas-tests/non-saas-table-mysql-bulk-delete-e2e.test.ts
  • backend/test/ava-tests/non-saas-tests/non-saas-table-postgres-bulk-delete-e2e.test.ts
  • backend/test/ava-tests/unit-tests/bulk-primary-keys-filter.test.ts
  • shared-code/src/data-access-layer/data-access-objects/data-access-object-clickhouse.ts
  • shared-code/src/data-access-layer/data-access-objects/data-access-object-mssql.ts
  • shared-code/src/data-access-layer/data-access-objects/data-access-object-mysql.ts
  • shared-code/src/data-access-layer/data-access-objects/data-access-object-oracle.ts
  • shared-code/src/data-access-layer/data-access-objects/data-access-object-postgres.ts
  • shared-code/src/helpers/bulk-primary-keys.ts
  • shared-code/src/helpers/errors/error-messages.ts
 _______________________________________________
< I'm approving this in spirit, but not in Git. >
 -----------------------------------------------
  \
   \   (\__/)
       (•ㅅ•)
       /   づ
✨ Finishing Touches
📝 Generate docstrings
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@Artuomka
Artuomka merged commit 903c7f4 into main Oct 14, 2026
16 of 18 checks passed
@Artuomka
Artuomka deleted the backend_bulk_delete_optimisation branch October 14, 2026 11:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant