Skip to content

refactor(audit): migrate the ClickHouse adapter to utopia-php/query 0.3 - #91

Open
lohanidamodar wants to merge 3 commits into
mainfrom
feat/audit-utopia-query-0.3.x
Open

refactor(audit): migrate the ClickHouse adapter to utopia-php/query 0.3#91
lohanidamodar wants to merge 3 commits into
mainfrom
feat/audit-utopia-query-0.3.x

Conversation

@lohanidamodar

@lohanidamodar lohanidamodar commented Jul 26, 2026

Copy link
Copy Markdown
Contributor

Supersedes utopia-php/audit#120.

utopia-php/audit is now a read-only subtree-split mirror of this monorepo, so the work has to land here. Per docs/distribution.md: "The distribution repositories become read-only mirrors: archive their open PRs, enable branch protection, and point contributors to the monorepo."

The mirror's origin/main was byte-identical to packages/audit/, so this is a direct transplant of that PR's net diff into packages/audit/, reconciled with monorepo layout and tooling.

What this does

Migrates the ClickHouse adapter from hand-assembled SQL to the utopia-php/query 0.3 schema/builder API.

  • setup() emits its DDL through Utopia\Query\Schema\ClickHouse: column types, LowCardinality(...) / Nullable(...) wrapping, bloom-filter indexes, engine, ORDER BY, PARTITION BY and SETTINGS all come from the schema builder. The retention MODIFY TTL / REMOVE TTL statements are unchanged.
  • find(), count(), getById(), createBatch() and cleanup() build their SQL through Utopia\Query\Builder\ClickHouse. Positional bindings become typed {paramN:Type} ClickHouse placeholders, derived from a column-to-type map built from getAttributes().
  • createBatch() uses bulkInsert(Format::JSONEachRow, ...) for the INSERT ... FORMAT JSONEachRow envelope and body instead of assembling the payload by hand.
  • cleanup() uses a lightweight DELETE FROM.
  • Query::getMethod() returns the Utopia\Query\Method enum upstream in 0.3, so Database::count() compares against enum cases. Utopia\Audit\Query keeps exposing the legacy TYPE_* string constants, which map to the same values.
  • Adds tests/Audit/Adapter/ClickHouseSqlSnapshotTest.php — a server-free SQL snapshot suite that pins the emitted DDL/INSERT/DELETE/SELECT shapes so a future query-lib bump cannot quietly change adapter SQL. It joins the unit testsuite and is excluded from e2e.

Filter semantics are unchanged: contains / notContains remain substring matches, now compiled to position(col, ?) > 0 / = 0 rather than LIKE '%needle%', which also removes the need for wildcard escaping.

Dependency bump

packages/audit/composer.json: utopia-php/query 0.1.* -> 0.3.*, locked at the tagged 0.3.3. No dev-branch pins. packages/audit/composer.lock regenerated; the result is byte-identical to the lock the mirror PR carried.

packages/audit is the only package in the monorepo that requires utopia-php/query, so the bump cannot conflict with a sibling.

Reconciliations with monorepo layout

  • Dropped the mirror PR's .gitignore change (adding /.phpunit.cache/) — the monorepo root .gitignore already ignores it.
  • Applied the monorepo's Rector rules to the ported code (bin/monorepo check audit --fix): ChangeOrIfContinueToMultiContinueRector in Database::count(), NewMethodCallWithoutParenthesesRector, and declare(strict_types=1) / final / assertSame on the new snapshot test.
  • No pint.json, phpstan.neon or CI workflow copies from the standalone repo were reintroduced; root pint.json / phpstan.neon stay the authority and packages/audit/phpstan.neon keeps its ../../phpstan.neon include. packages/audit/.github/workflows/mirror.yml is untouched.
  • docker-compose.yml unchanged; phpunit.xml only gains the snapshot-test entries in the existing unit / e2e testsuites.

Test plan

Run on PHP 8.5.8 with the monorepo toolchain.

  • bin/monorepo check audit — pint passed, PHPStan [OK] No errors, Rector [OK] Rector is done!, all checks passed.
  • bin/monorepo validateall packages valid.
  • packages/audit unit suite (composer test, i.e. phpunit --testsuite unit) — Tests: 22, Assertions: 106, Deprecations: 3, no failures. The three deprecations are Utopia\Query\Query::contains() in 0.3 pointing at containsString() / containsAny(); behaviour is unchanged and switching call sites is out of scope here.

The e2e tier could not run locally — no Docker daemon was available, so composer test:e2e and its compose services (ClickHouse, MariaDB) could not be started. Utopia\Tests\Audit\Adapter\ClickHouseTest and Utopia\Tests\Audit\Adapter\DatabaseTest were therefore unverified on my machine.

CI has since covered that gap: the test (audit) job brought up clickhouse/clickhouse-server:25.11-alpine and mariadb:10.11 and ran the e2e suite green — Tests: 91, Assertions: 979, Deprecations: 8, no failures.

Note on the mirror

utopia-php/audit's feat/utopia-query-0.3.x branch is deliberately left in place at c1aefabappwrite-labs/cloud#3965 currently pins utopia-php/audit: dev-feat/utopia-query-0.3.x and its lock references that commit. Only the PR is closed.

Moves the audit package from utopia-php/query 0.1.* to the 0.3 line
(locked at 0.3.3) and refreshes packages/audit/composer.lock.
setup() emits its DDL through Utopia\Query\Schema\ClickHouse instead of
hand-assembled SQL: column types, LowCardinality/Nullable wrapping,
bloom-filter indexes, engine, ORDER BY, PARTITION BY and SETTINGS all come
from the schema builder. The retention MODIFY TTL / REMOVE TTL statements
are unchanged.

find(), count(), getById(), createBatch() and cleanup() build their SQL
through Utopia\Query\Builder\ClickHouse. Positional bindings become typed
{paramN:Type} placeholders derived from getAttributes(). createBatch() uses
bulkInsert(Format::JSONEachRow, ...) for the INSERT envelope and body, and
cleanup() uses a lightweight DELETE FROM.

Query::getMethod() now returns the Utopia\Query\Method enum upstream, so
Database::count() compares against enum cases; Utopia\Audit\Query keeps
exposing the legacy TYPE_* string constants.

Filter semantics are unchanged: contains/notContains stay substring matches,
now compiled to position(col, ?) > 0 / = 0 instead of LIKE '%needle%', which
also drops the wildcard escaping.

Adds tests/Audit/Adapter/ClickHouseSqlSnapshotTest.php, a server-free SQL
snapshot suite pinning the emitted DDL/INSERT/DELETE/SELECT shapes; it runs
in the unit testsuite and is excluded from e2e.
@greptile-apps

greptile-apps Bot commented Jul 26, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR migrates the audit package’s ClickHouse SQL generation to utopia-php/query 0.3.

  • Replaces hand-built DDL and query strings with ClickHouse schema and query builders.
  • Adds typed named bindings and builder-generated JSONEachRow inserts and lightweight deletes.
  • Updates query-method handling to use the upstream Method enum.
  • Adds server-free SQL-shape tests and updates the audit dependency lockfile.

Confidence Score: 5/5

The PR appears safe to merge, with only a non-blocking test-coverage concern remaining.

No blocking failure remains.

Important Files Changed

Filename Overview
packages/audit/src/Audit/Adapter/ClickHouse.php Migrates schema creation, filtering, selection, counting, insertion, and cleanup SQL to the query 0.3 ClickHouse builders.
packages/audit/src/Audit/Query.php Adapts the audit query wrapper to the upstream Method enum while retaining legacy string constants.
packages/audit/src/Audit/Adapter/Database.php Updates count-query method comparisons for enum-returning query objects.
packages/audit/tests/Audit/Adapter/ClickHouseSqlSnapshotTest.php Adds unit coverage for representative SQL shapes produced by the query library’s ClickHouse schema and statement builders.
packages/audit/composer.json Raises the audit package’s utopia-php/query requirement from 0.1 to 0.3.
packages/audit/composer.lock Locks utopia-php/query at version 0.3.3.

Reviews (2): Last reviewed commit: "Merge remote-tracking branch 'origin/mai..." | Re-trigger Greptile

Comment on lines +53 to +54
$schema = new ClickHouseSchema();
$table = $schema->table('default.audits');

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Snapshots bypass adapter integration

These snapshots directly reconstruct the schema and query-builder calls instead of invoking the adapter methods they claim to cover. An adapter-level omission, incorrect argument, or binding-merge regression therefore leaves the suite green while production emits different SQL or parameters.

Prompt To Fix With AI
This is a comment left during a code review.
Path: packages/audit/tests/Audit/Adapter/ClickHouseSqlSnapshotTest.php
Line: 53-54

Comment:
**Snapshots bypass adapter integration**

These snapshots directly reconstruct the schema and query-builder calls instead of invoking the adapter methods they claim to cover. An adapter-level omission, incorrect argument, or binding-merge regression therefore leaves the suite green while production emits different SQL or parameters.

How can I resolve this? If you propose a fix, please make it concise.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

Fix in Claude Code Fix in Codex

…ry-0.3.x

# Conflicts:
#	packages/audit/composer.lock
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant