Skip to content

BE-742: Keep the messages Postgres sends outside a statement's results - #9636

Open
TimDiekmann wants to merge 5 commits into
mainfrom
t/be-742-keep-the-messages-postgres-sends-outside-a-statements
Open

BE-742: Keep the messages Postgres sends outside a statement's results#9636
TimDiekmann wants to merge 5 commits into
mainfrom
t/be-742-keep-the-messages-postgres-sends-outside-a-statements

Conversation

@TimDiekmann

@TimDiekmann TimDiekmann commented Sep 9, 2026

Copy link
Copy Markdown
Member

🌟 What is the purpose of this PR?

Postgres reports warnings and notices on the same wire as query results, but tokio_postgres delivers them on a side channel that only Connection::poll_message exposes. Awaiting the connection, which is what a pool does by default, discards every one of them. Server warnings have therefore been invisible for as long as the graph has had a pool: SET LOCAL outside a transaction, for one, is only ever reported as a warning.

The graph now owns its pool's connection manager. Each connection runs a task that reads that channel, so what the server says becomes a tracing event under hash_graph_postgres_store::server at the level of its severity, with the SQLSTATE, detail, hint and context as fields. A termination the server initiates arrives on the same channel as a fatal error response and is recorded the same way, and a connection whose recording task has stopped is not handed out again.

On the way, the store gets a transaction API of its own instead of implementing the migration runner's traits, and deadpool-postgres goes.

🔗 Related links

  • BE-742 (internal)
  • Parent: BE-708 (internal), plan capture over the same channel, which this PR deliberately leaves out
  • Follow-up: BE-703 (internal), pool timeouts

🚫 Blocked by

  • Nothing.

🔍 What does this change?

  • Store transaction API (first commit): PostgresStore::transaction, the builder's isolation_level/read_only/deferrable, and commit/rollback are inherent methods. The store no longer implements hash_graph_migrations::{Context, Transaction, TransactionBuilder}, and nine test and bench files stop importing those traits to open a transaction.
  • Migrations crate: the runner never configured a transaction, so TransactionBuilder folds into Context::transaction and IsolationLevel moves to hash-graph-postgres-store next to TransactionOptions. hash-graph-postgres-store no longer depends on hash-graph-migrations; package.json, docs/task-dependencies.json and yarn.lock are regenerated accordingly.
  • connection.rs (new): ManagedConnection pairs a tokio_postgres::Client with the task driving poll_message; ConnectionManager implements deadpool::managed::Manager and rejects a connection on recycle when its client is closed or its task has finished. ConnectionError replaces deadpool_postgres::PoolError as StorePool::Error.
  • pool.rs: PooledConnection is a newtype over deadpool's Object, which keeps deadpool a private dependency. PostgresStorePool::new never connected and is no longer async; the callers drop their .await.
  • deadpool-postgres is removed. Only its Config was ever used: the statement cache is opt-in and was never opted into, and its transaction wrappers exist to offer that same cache.
  • The SET LOCAL sites in the store already run inside transactions, so the new warning does not fire on a hot path.

Pre-Merge Checklist 🚀

🚢 Has this modified a publishable library?

This PR:

  • does not modify any publishable blocks or libraries, or modifications do not need publishing

📜 Does this require a change to the docs?

The changes in this PR:

  • are internal and do not require a docs change

🕸️ Does this require a change to the Turbo Graph?

The changes in this PR:

  • affected the execution graph, and the turbo.json's have been updated to reflect this
    • hash-graph-postgres-store loses its edge to hash-graph-migrations; mise run sync:turborepo regenerated package.json and docs/task-dependencies.json, no turbo.json needed a change

⚠️ Known issues

  • Pool timeouts stay unbounded, as before: deadpool's defaults are all None, matching the explicit Nones the old configuration set. ConnectionError::Unavailable therefore covers a wait timeout and a missing runtime with one variant, and no pool error carries a StatusCode, so a saturated pool would answer 500 rather than 503. Unreachable until a timeout is configured; tracked in BE-703, where the variant split and the status code belong.
  • Server messages carry no request trace. The recording task runs outside any span on purpose, because a connection outlives the request that made the pool grow. Every event carries the connection id as a field of its own, so it survives any filter level, and that id joins it to the acquisition that created the connection, which is logged within that request's span. Attributing a message to the statement that provoked it needs a collector, which is BE-708's subject.
  • Session state is not reset when a connection returns to the pool, unchanged from deadpool-postgres's RecyclingMethod::Fast. The store sets session parameters only with SET LOCAL inside a transaction, and a SET LOCAL outside one now shows up as a warning, so a reset round trip per checkout is not worth its cost.

🐾 Next steps

  • BE-708: plan capture via auto_explain notices over this channel, with a collector per transaction.
  • BE-741: turning plan capture on in a deployment without a redeploy.
  • BE-703: pool timeouts, with the error split and the status code above.

🛡 What tests cover this?

  • New, in libs/@local/graph/postgres-store/tests/connection/main.rs, each installing a tracing layer that records what reaches hash_graph_postgres_store::server:
    • connection_records_server_warnings: SET LOCAL outside a transaction is recorded at WARN with severity, SQLSTATE 25P01 and the connection id on the event.
    • connection_records_server_notices: RAISE NOTICE … USING DETAIL, HINT is recorded at INFO with detail, hint and context, so the level mapping is pinned on more than one arm and every field the server sends is asserted.
    • connection_records_notifications: LISTEN and NOTIFY on one connection record the notification at INFO with its channel.
    • connection_records_server_termination: pg_terminate_backend from a second connection is recorded at ERROR with FATAL and 57P01.
    • pool_replaces_terminated_connection: a pool of one connection does not hand a terminated backend out again.
    • pool_reports_unreachable_database: the pool connects lazily and reports a missing database as ConnectionError::Connect with the driver's error in the report.
  • Existing, in tests/graph/integration/postgres/transaction.rs: options composition, defaults and savepoint nesting through the store's transaction API.
  • Every other integration test reaches the database through the new pool.

❓ How to test this?

  1. Start the compose stack and run cargo nextest run -p hash-graph-postgres-store --all-features --test connection.
  2. Run the graph with RUST_LOG=hash_graph_postgres_store::server=info and, from psql, SELECT pg_terminate_backend(pid) FROM pg_stat_activity WHERE usename = 'graph' AND state = 'idle' LIMIT 1;.
  3. Confirm the graph logs an ERROR event with severity=FATAL and code=57P01, and that the next request succeeds on a fresh connection.

📹 Demo

No UI. The events the tests assert on are what an operator sees in the graph's log.

TimDiekmann and others added 3 commits September 9, 2026 18:19
`Context`, `TransactionBuilder` and `Transaction` belong to the migration
runner, which is generic over contexts and reaches a transaction through
them. The store implemented them because it runs migrations, but then had no
transaction API of its own, so every ordinary read and write in the data path
went through the migration runner's contract. Opening a transaction meant
importing a migrations trait, which nine test and bench files did.

The setters and the entry point are now the store's own. The runner never
configured a transaction, so `TransactionBuilder` loses its options and folds
into `Context::transaction`, and `IsolationLevel` moves next to the options
that use it. The store no longer depends on the migrations crate.

Co-authored-by: Claude <noreply@anthropic.com>
`tokio_postgres` delivers warnings, notices and notifications on a side
channel that only `Connection::poll_message` exposes, and awaiting the
connection — which is what a pool does by default — discards every one of
them. Server warnings were therefore invisible: `SET LOCAL` outside a
transaction, for one, only ever says so in a warning.

`ManagedConnection` drives that channel from a task of its own, so each
message becomes a tracing event under `hash_graph_postgres_store::server` at
the level of its severity, with the SQLSTATE, detail, hint and context as
fields. A termination the server initiates arrives on the same channel as a
fatal error response and is recorded the same way. A connection whose
recording task has stopped counts as closed, so the pool does not hand it out
again.

This replaces `deadpool-postgres` with a manager over `deadpool` itself. Only
`Config` was ever used from it: the statement cache is opt-in and was never
opted into, and the transaction wrappers exist to offer that same cache.
`PostgresStorePool::new` never connected, so it is no longer async.

Co-authored-by: Claude <noreply@anthropic.com>
@TimDiekmann TimDiekmann self-assigned this Sep 9, 2026
@vercel

vercel Bot commented Sep 9, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

4 Skipped Deployments
Project Deployment Actions Updated
hash Ignored Ignored Preview Sep 10, 2026 11:03am UTC
hashdotdesign-tokens Ignored Ignored Preview Sep 10, 2026 11:03am UTC
petrinaut Skipped Skipped Sep 10, 2026 11:03am UTC
petrinaut-docs Skipped Skipped Sep 10, 2026 11:03am UTC

Request Review

@github-actions github-actions Bot added area/deps Relates to third-party dependencies (area) area/apps > hash* Affects HASH (a `hash-*` app) area/libs Relates to first-party libraries/crates/packages (area) type/eng > backend Owned by the @backend team area/tests New or updated tests area/apps area/apps > hash-graph labels Sep 9, 2026
@codecov

codecov Bot commented Sep 9, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 0% with 144 lines in your changes missing coverage. Please review.
✅ Project coverage is 65.86%. Comparing base (407734a) to head (11acaae).
⚠️ Report is 12 commits behind head on main.

Files with missing lines Patch % Lines
...ph/postgres-store/src/store/postgres/connection.rs 0.00% 109 Missing ⚠️
...al/graph/postgres-store/src/store/postgres/pool.rs 0.00% 29 Missing ⚠️
...cal/graph/postgres-store/src/store/postgres/mod.rs 0.00% 6 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #9636      +/-   ##
==========================================
- Coverage   65.89%   65.86%   -0.04%     
==========================================
  Files        1887     1888       +1     
  Lines      198443   198559     +116     
  Branches     8248     8248              
==========================================
- Hits       130773   130772       -1     
- Misses      66140    66257     +117     
  Partials     1530     1530              
Flag Coverage Δ
apps.hash-ai-worker-ts 1.99% <ø> (ø)
apps.hash-api 15.35% <ø> (ø)
apps.hash-graph 12.62% <ø> (+0.08%) ⬆️
blockprotocol.type-system 38.15% <ø> (ø)
local.claude-hooks 0.00% <ø> (ø)
local.harpc-client 51.49% <ø> (ø)
local.hash-backend-utils 3.27% <ø> (ø)
local.hash-graph-sdk 10.02% <ø> (ø)
local.hash-isomorphic-utils 12.22% <ø> (ø)
rust.antsi 2.36% <ø> (ø)
rust.error-stack 90.81% <ø> (ø)
rust.harpc-codec 84.70% <ø> (ø)
rust.harpc-net 96.19% <ø> (-0.02%) ⬇️
rust.harpc-tower 67.03% <ø> (ø)
rust.harpc-types 0.00% <ø> (ø)
rust.harpc-wire-protocol 92.23% <ø> (ø)
rust.hash-codec 72.76% <ø> (ø)
rust.hash-config 81.14% <ø> (ø)
rust.hash-graph-api 19.71% <ø> (ø)
rust.hash-graph-atlas 80.36% <ø> (ø)
rust.hash-graph-authentication 96.02% <ø> (ø)
rust.hash-graph-authorization 63.14% <ø> (ø)
rust.hash-graph-embeddings 91.88% <ø> (ø)
rust.hash-graph-postgres-store 31.96% <0.00%> (-0.19%) ⬇️
rust.hash-graph-store 48.41% <ø> (ø)
rust.hash-graph-temporal-versioning 50.18% <ø> (ø)
rust.hash-graph-types 0.00% <ø> (ø)
rust.hash-graph-validation 84.71% <ø> (ø)
rust.hash-middleware 90.92% <ø> (ø)
rust.hashql-ast 89.63% <ø> (ø)
rust.hashql-compiletest 28.39% <ø> (ø)
rust.hashql-core 78.92% <ø> (ø)
rust.hashql-diagnostics 72.51% <ø> (ø)
rust.hashql-eval 79.82% <ø> (ø)
rust.hashql-hir 89.09% <ø> (ø)
rust.hashql-mir 87.92% <ø> (ø)
rust.hashql-syntax-jexpr 94.04% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@codspeed-hq

codspeed-hq Bot commented Sep 9, 2026

Copy link
Copy Markdown

Merging this PR will not alter performance

⚠️ 6 benchmarks measured no execution time

Nothing ran under measurement, usually because the compiler removed the code under test. These results are not comparable, so they count as unchanged.

Preventing compiler optimizations

✅ 98 untouched benchmarks

Performance Changes

Benchmark BASE HEAD Efficiency
⚠️ as_constant < 1 ns < 1 ns N/A
⚠️ constant_equal < 1 ns < 1 ns N/A
⚠️ constant_not_equal < 1 ns < 1 ns N/A
⚠️ access < 1 ns < 1 ns N/A
⚠️ runtime_equal < 1 ns < 1 ns N/A
⚠️ runtime_not_equal < 1 ns < 1 ns N/A

Comparing t/be-742-keep-the-messages-postgres-sends-outside-a-statements (11acaae) with main (4c664c0)1

Open in CodSpeed

Footnotes

  1. No successful run was found on main (890b8c4) during the generation of this report, so 4c664c0 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report.

Comment thread libs/@local/graph/postgres-store/tests/common/mod.rs Dismissed
Comment thread tests/graph/benches/manual_queries/entity_queries/mod.rs Dismissed
@vercel
vercel Bot temporarily deployed to Preview – petrinaut-docs September 10, 2026 07:59 Inactive
@vercel
vercel Bot temporarily deployed to Preview – petrinaut September 10, 2026 07:59 Inactive
@TimDiekmann
TimDiekmann marked this pull request as ready for review September 10, 2026 10:47
@TimDiekmann
TimDiekmann requested a review from a team as a code owner September 10, 2026 10:47
@TimDiekmann
TimDiekmann requested review from a team and a balanced review from Copilot September 10, 2026 10:47
@cursor

cursor Bot commented Sep 10, 2026

Copy link
Copy Markdown

PR Summary

Medium Risk
Refactors the core Postgres connection pool and transaction API used by every graph CLI, server, and test; behavior changes include lazy connect, new error types, and decoupling from the migrations crate.

Overview
Replaces deadpool-postgres with a custom deadpool::managed pool so each checkout keeps a background task on tokio_postgres::Connection::poll_message. Postgres warnings, notices, notifications, and server-initiated terminations are emitted as tracing events under hash_graph_postgres_store::server (with SQLSTATE, detail, hint, context) instead of being dropped when the connection is driven only for pooling.

PostgresStorePool::new is now synchronous and connects lazily on acquire; pool failures surface as ConnectionError instead of deadpool_postgres::PoolError. PooledConnection / ManagedConnection wrap the client and recycle rejects closed backends.

hash-graph-postgres-store no longer depends on hash-graph-migrations. Transaction control moves onto PostgresStore (transaction() builder, commit / rollback, IsolationLevel in the store crate); the migrations crate’s Context API is simplified to a plain async fn transaction() without TransactionBuilder. Call sites drop .await on pool construction and stop importing migration transaction traits.

Adds tests/connection coverage for warning/notice/notification/termination logging and pool recycle behavior; updates workspace lockfiles and dependency diagrams accordingly.

Reviewed by Cursor Bugbot for commit 11acaae. Bugbot is set up for automated code reviews on this repo. Configure here.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

WARN-level filtering drops connection correlation, and key preserved message fields remain untested.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

This PR replaces deadpool-postgres with a custom managed PostgreSQL pool that records asynchronous server messages through tracing and exposes native store transaction APIs.

Changes:

  • Adds managed connections that record notices, warnings, notifications, and termination errors.
  • Moves transaction configuration into PostgresStore and simplifies migration traits.
  • Updates dependencies, callers, tests, benchmarks, and generated dependency metadata.
File summaries
File Description
yarn.lock Regenerates workspace dependency wiring.
tests/graph/integration/postgres/transaction.rs Uses inherent transaction APIs.
tests/graph/integration/postgres/lib.rs Adapts synchronous pool construction.
tests/graph/benches/util.rs Updates benchmark pool construction.
tests/graph/benches/representative_read/seed.rs Removes migration trait imports.
tests/graph/benches/read_scaling/knowledge/linkless/entity.rs Removes migration trait imports.
tests/graph/benches/read_scaling/knowledge/complete/entity.rs Removes migration trait imports.
tests/graph/benches/policy/seed.rs Uses inherent transaction methods.
tests/graph/benches/manual_queries/entity_queries/mod.rs Removes unnecessary runtime blocking.
tests/graph/benches/graph/scenario/runner.rs Adapts synchronous pool construction.
libs/@local/telemetry/docs/dependency-diagram.mmd Regenerates dependency graph.
libs/@local/hashql/syntax-jexpr/docs/dependency-diagram.mmd Regenerates dependency graph.
libs/@local/hashql/mir/docs/dependency-diagram.mmd Regenerates dependency graph.
libs/@local/hashql/hir/docs/dependency-diagram.mmd Regenerates dependency graph.
libs/@local/hashql/eval/docs/dependency-diagram.mmd Regenerates dependency graph.
libs/@local/hashql/compiletest/docs/dependency-diagram.mmd Regenerates dependency graph.
libs/@local/hashql/ast/docs/dependency-diagram.mmd Regenerates dependency graph.
libs/@local/graph/postgres-store/tests/principals/main.rs Removes obsolete trait import.
libs/@local/graph/postgres-store/tests/principals/actions.rs Removes obsolete trait import.
libs/@local/graph/postgres-store/tests/deletion/main.rs Removes obsolete trait import.
libs/@local/graph/postgres-store/tests/connection/main.rs Tests server-message recording and recycling.
libs/@local/graph/postgres-store/tests/common/mod.rs Extracts reusable connection configuration.
libs/@local/graph/postgres-store/src/store/postgres/query/compile/peephole/tuple.rs Updates statement-cache documentation.
libs/@local/graph/postgres-store/src/store/postgres/pool.rs Implements the custom managed pool wrapper.
libs/@local/graph/postgres-store/src/store/postgres/ontology/property_type.rs Removes obsolete trait import.
libs/@local/graph/postgres-store/src/store/postgres/ontology/entity_type.rs Removes obsolete trait import.
libs/@local/graph/postgres-store/src/store/postgres/ontology/data_type.rs Removes obsolete trait import.
libs/@local/graph/postgres-store/src/store/postgres/mod.rs Adds inherent transaction APIs and exports.
libs/@local/graph/postgres-store/src/store/postgres/knowledge/entity/mod.rs Removes obsolete trait import.
libs/@local/graph/postgres-store/src/store/postgres/connection.rs Drives and records PostgreSQL side-channel messages.
libs/@local/graph/postgres-store/src/store/mod.rs Updates the store’s public exports.
libs/@local/graph/postgres-store/src/snapshot/mod.rs Removes obsolete trait import.
libs/@local/graph/postgres-store/src/permissions/mod.rs Removes obsolete trait import.
libs/@local/graph/postgres-store/src/lib.rs Enables trait aliases.
libs/@local/graph/postgres-store/package.json Removes generated migrations dependency wiring.
libs/@local/graph/postgres-store/docs/task-dependencies.json Updates generated task dependencies.
libs/@local/graph/postgres-store/docs/dependency-diagram.mmd Regenerates dependency graph.
libs/@local/graph/postgres-store/Cargo.toml Replaces deadpool-postgres dependencies and features.
libs/@local/graph/migrations/src/postgres.rs Simplifies PostgreSQL migration transactions.
libs/@local/graph/migrations/src/lib.rs Removes transaction-builder exports.
libs/@local/graph/migrations/src/context.rs Folds transaction creation into Context.
libs/@local/graph/migrations/docs/dependency-diagram.mmd Regenerates dependency graph.
libs/@local/graph/migrations-macros/docs/dependency-diagram.mmd Regenerates dependency graph.
libs/@local/graph/atlas/tests/route_fixture.rs Adapts synchronous pool construction.
libs/@local/graph/atlas/docs/dependency-diagram.mmd Regenerates dependency graph.
libs/@local/graph/api/docs/dependency-diagram.mmd Regenerates dependency graph.
Cargo.toml Removes the workspace deadpool-postgres dependency.
Cargo.lock Regenerates Rust dependency resolution.
apps/hash-graph/src/subcommand/snapshot.rs Adapts synchronous pool construction.
apps/hash-graph/src/subcommand/server.rs Adapts synchronous pool construction.
apps/hash-graph/src/subcommand/reindex_cache.rs Adapts synchronous pool construction.
apps/hash-graph/src/subcommand/migrate.rs Adapts synchronous pool construction.
apps/hash-graph/src/subcommand/atlas.rs Adapts synchronous pool construction.
apps/hash-graph/src/subcommand/admin_server.rs Adapts synchronous pool construction.
apps/hash-graph/docs/dependency-diagram.mmd Regenerates dependency graph.
Review details

Suppressed comments (1)

libs/@local/graph/postgres-store/src/store/postgres/connection.rs:263

  • An INFO span is disabled when the active filter is WARN (for example HASH_GRAPH_LOG_LEVEL=warn), so the WARN/ERROR server events emitted inside this instrumented future lose the span's connection field. That breaks the advertised correlation exactly at a normal production threshold. Thread id through drive/record/report, record it directly on every server event, and test this with a WARN-filtered subscriber.
            let span = tracing::info_span!(parent: None, "postgres_connection", connection = id);
  • Files reviewed: 53/55 changed files
  • Comments generated: 1
  • Review effort level: Balanced

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread libs/@local/graph/postgres-store/src/store/postgres/connection.rs
… span

An INFO span is disabled under a WARN filter, and the WARN and ERROR events
recorded inside it lose the span's fields with it. The connection id is a
field of each event now, so it survives whatever filter is active.
@github-actions

Copy link
Copy Markdown
Contributor

Benchmark results

@rust/hash-graph-benches – Integrations

policy_resolution_large

Function Value Mean Flame graphs
resolve_policies_for_actor user: empty, selectivity: high, policies: 2002 $$27.8 \mathrm{ms} \pm 238 \mathrm{μs}\left({\color{gray}0.250 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: empty, selectivity: low, policies: 1 $$3.45 \mathrm{ms} \pm 25.6 \mathrm{μs}\left({\color{gray}-0.448 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: empty, selectivity: medium, policies: 1002 $$12.8 \mathrm{ms} \pm 125 \mathrm{μs}\left({\color{gray}1.19 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: high, policies: 3314 $$43.5 \mathrm{ms} \pm 395 \mathrm{μs}\left({\color{gray}0.255 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: low, policies: 1 $$14.3 \mathrm{ms} \pm 131 \mathrm{μs}\left({\color{gray}1.23 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: medium, policies: 1527 $$24.4 \mathrm{ms} \pm 238 \mathrm{μs}\left({\color{gray}1.49 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: high, policies: 2078 $$28.6 \mathrm{ms} \pm 232 \mathrm{μs}\left({\color{gray}-0.177 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: low, policies: 1 $$3.78 \mathrm{ms} \pm 29.5 \mathrm{μs}\left({\color{gray}1.44 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: medium, policies: 1033 $$13.9 \mathrm{ms} \pm 129 \mathrm{μs}\left({\color{gray}0.925 \mathrm{\%}}\right) $$ Flame Graph

policy_resolution_medium

Function Value Mean Flame graphs
resolve_policies_for_actor user: empty, selectivity: high, policies: 102 $$3.85 \mathrm{ms} \pm 35.3 \mathrm{μs}\left({\color{gray}-1.326 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: empty, selectivity: low, policies: 1 $$3.08 \mathrm{ms} \pm 24.5 \mathrm{μs}\left({\color{gray}0.400 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: empty, selectivity: medium, policies: 52 $$3.47 \mathrm{ms} \pm 25.4 \mathrm{μs}\left({\color{gray}1.49 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: high, policies: 269 $$5.29 \mathrm{ms} \pm 43.8 \mathrm{μs}\left({\color{gray}-0.242 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: low, policies: 1 $$3.60 \mathrm{ms} \pm 22.5 \mathrm{μs}\left({\color{gray}-1.773 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: medium, policies: 108 $$4.24 \mathrm{ms} \pm 31.3 \mathrm{μs}\left({\color{gray}-0.435 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: high, policies: 133 $$4.58 \mathrm{ms} \pm 36.9 \mathrm{μs}\left({\color{gray}1.17 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: low, policies: 1 $$3.50 \mathrm{ms} \pm 27.9 \mathrm{μs}\left({\color{gray}-0.360 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: medium, policies: 63 $$4.25 \mathrm{ms} \pm 32.3 \mathrm{μs}\left({\color{gray}1.35 \mathrm{\%}}\right) $$ Flame Graph

policy_resolution_none

Function Value Mean Flame graphs
resolve_policies_for_actor user: empty, selectivity: high, policies: 2 $$2.76 \mathrm{ms} \pm 18.7 \mathrm{μs}\left({\color{gray}-0.089 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: empty, selectivity: low, policies: 1 $$2.74 \mathrm{ms} \pm 18.0 \mathrm{μs}\left({\color{gray}1.25 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: empty, selectivity: medium, policies: 2 $$2.85 \mathrm{ms} \pm 19.2 \mathrm{μs}\left({\color{gray}1.43 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: high, policies: 8 $$3.12 \mathrm{ms} \pm 21.3 \mathrm{μs}\left({\color{gray}1.42 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: low, policies: 1 $$2.93 \mathrm{ms} \pm 26.9 \mathrm{μs}\left({\color{gray}1.28 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: medium, policies: 3 $$3.21 \mathrm{ms} \pm 24.7 \mathrm{μs}\left({\color{gray}1.02 \mathrm{\%}}\right) $$ Flame Graph

policy_resolution_small

Function Value Mean Flame graphs
resolve_policies_for_actor user: empty, selectivity: high, policies: 52 $$3.13 \mathrm{ms} \pm 26.1 \mathrm{μs}\left({\color{gray}-0.154 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: empty, selectivity: low, policies: 1 $$2.87 \mathrm{ms} \pm 23.2 \mathrm{μs}\left({\color{gray}1.77 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: empty, selectivity: medium, policies: 26 $$2.99 \mathrm{ms} \pm 17.8 \mathrm{μs}\left({\color{gray}0.094 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: high, policies: 94 $$3.59 \mathrm{ms} \pm 27.6 \mathrm{μs}\left({\color{gray}-0.066 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: low, policies: 1 $$3.15 \mathrm{ms} \pm 27.1 \mathrm{μs}\left({\color{gray}2.72 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: medium, policies: 27 $$3.39 \mathrm{ms} \pm 25.1 \mathrm{μs}\left({\color{gray}0.343 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: high, policies: 66 $$3.57 \mathrm{ms} \pm 30.7 \mathrm{μs}\left({\color{gray}1.74 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: low, policies: 1 $$3.12 \mathrm{ms} \pm 22.1 \mathrm{μs}\left({\color{gray}1.16 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: medium, policies: 29 $$3.44 \mathrm{ms} \pm 23.4 \mathrm{μs}\left({\color{gray}0.298 \mathrm{\%}}\right) $$ Flame Graph

read_scaling_complete

Function Value Mean Flame graphs
entity_by_id;one_depth 1 entities $$32.3 \mathrm{ms} \pm 206 \mathrm{μs}\left({\color{gray}1.50 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;one_depth 10 entities $$71.4 \mathrm{ms} \pm 620 \mathrm{μs}\left({\color{gray}1.59 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;one_depth 25 entities $$36.1 \mathrm{ms} \pm 250 \mathrm{μs}\left({\color{gray}3.04 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;one_depth 5 entities $$39.7 \mathrm{ms} \pm 248 \mathrm{μs}\left({\color{gray}0.814 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;one_depth 50 entities $$41.8 \mathrm{ms} \pm 227 \mathrm{μs}\left({\color{gray}-3.873 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;two_depth 1 entities $$33.9 \mathrm{ms} \pm 193 \mathrm{μs}\left({\color{gray}0.481 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;two_depth 10 entities $$418 \mathrm{ms} \pm 1.41 \mathrm{ms}\left({\color{gray}0.404 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;two_depth 25 entities $$91.5 \mathrm{ms} \pm 709 \mathrm{μs}\left({\color{gray}-0.115 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;two_depth 5 entities $$78.9 \mathrm{ms} \pm 594 \mathrm{μs}\left({\color{gray}-0.209 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;two_depth 50 entities $$277 \mathrm{ms} \pm 1.22 \mathrm{ms}\left({\color{gray}1.11 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;zero_depth 1 entities $$10.6 \mathrm{ms} \pm 79.2 \mathrm{μs}\left({\color{gray}1.53 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;zero_depth 10 entities $$10.9 \mathrm{ms} \pm 64.9 \mathrm{μs}\left({\color{gray}3.37 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;zero_depth 25 entities $$10.8 \mathrm{ms} \pm 66.4 \mathrm{μs}\left({\color{gray}0.375 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;zero_depth 5 entities $$10.8 \mathrm{ms} \pm 57.4 \mathrm{μs}\left({\color{gray}2.01 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;zero_depth 50 entities $$10.8 \mathrm{ms} \pm 90.7 \mathrm{μs}\left({\color{gray}1.78 \mathrm{\%}}\right) $$ Flame Graph

read_scaling_linkless

Function Value Mean Flame graphs
entity_by_id 1 entities $$10.6 \mathrm{ms} \pm 86.0 \mathrm{μs}\left({\color{gray}0.792 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 10 entities $$10.6 \mathrm{ms} \pm 59.6 \mathrm{μs}\left({\color{gray}0.448 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 100 entities $$10.7 \mathrm{ms} \pm 87.0 \mathrm{μs}\left({\color{gray}2.20 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 1000 entities $$10.8 \mathrm{ms} \pm 63.2 \mathrm{μs}\left({\color{gray}4.27 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 10000 entities $$10.9 \mathrm{ms} \pm 71.6 \mathrm{μs}\left({\color{gray}0.143 \mathrm{\%}}\right) $$ Flame Graph

representative_read_entity

Function Value Mean Flame graphs
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/block/v/1 $$11.6 \mathrm{ms} \pm 70.3 \mathrm{μs}\left({\color{gray}4.25 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/book/v/1 $$11.4 \mathrm{ms} \pm 73.8 \mathrm{μs}\left({\color{gray}4.37 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/building/v/1 $$11.6 \mathrm{ms} \pm 72.9 \mathrm{μs}\left({\color{red}6.09 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/organization/v/1 $$11.4 \mathrm{ms} \pm 75.4 \mathrm{μs}\left({\color{gray}4.49 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/page/v/2 $$11.5 \mathrm{ms} \pm 101 \mathrm{μs}\left({\color{gray}4.64 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/person/v/1 $$11.2 \mathrm{ms} \pm 68.3 \mathrm{μs}\left({\color{gray}2.94 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/playlist/v/1 $$11.7 \mathrm{ms} \pm 83.0 \mathrm{μs}\left({\color{red}6.77 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/song/v/1 $$11.3 \mathrm{ms} \pm 68.2 \mathrm{μs}\left({\color{gray}2.39 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/uk-address/v/1 $$11.8 \mathrm{ms} \pm 93.2 \mathrm{μs}\left({\color{red}6.81 \mathrm{\%}}\right) $$ Flame Graph

representative_read_entity_type

Function Value Mean Flame graphs
get_entity_type_by_id Account ID: bf5a9ef5-dc3b-43cf-a291-6210c0321eba $$8.34 \mathrm{ms} \pm 55.0 \mathrm{μs}\left({\color{gray}2.76 \mathrm{\%}}\right) $$ Flame Graph

representative_read_multiple_entities

Function Value Mean Flame graphs
entity_by_property traversal_paths=0 0 $$57.1 \mathrm{ms} \pm 396 \mathrm{μs}\left({\color{gray}3.23 \mathrm{\%}}\right) $$
entity_by_property traversal_paths=255 1,resolve_depths=inherit:1;values:255;properties:255;links:127;link_dests:126;type:true $$111 \mathrm{ms} \pm 701 \mathrm{μs}\left({\color{gray}3.21 \mathrm{\%}}\right) $$
entity_by_property traversal_paths=2 1,resolve_depths=inherit:0;values:0;properties:0;links:0;link_dests:0;type:false $$63.2 \mathrm{ms} \pm 440 \mathrm{μs}\left({\color{gray}3.41 \mathrm{\%}}\right) $$
entity_by_property traversal_paths=2 1,resolve_depths=inherit:0;values:0;properties:0;links:1;link_dests:0;type:true $$72.6 \mathrm{ms} \pm 527 \mathrm{μs}\left({\color{gray}2.58 \mathrm{\%}}\right) $$
entity_by_property traversal_paths=2 1,resolve_depths=inherit:0;values:0;properties:2;links:1;link_dests:0;type:true $$81.8 \mathrm{ms} \pm 418 \mathrm{μs}\left({\color{gray}2.80 \mathrm{\%}}\right) $$
entity_by_property traversal_paths=2 1,resolve_depths=inherit:0;values:2;properties:2;links:1;link_dests:0;type:true $$89.2 \mathrm{ms} \pm 766 \mathrm{μs}\left({\color{gray}2.93 \mathrm{\%}}\right) $$
link_by_source_by_property traversal_paths=0 0 $$46.9 \mathrm{ms} \pm 284 \mathrm{μs}\left({\color{gray}3.59 \mathrm{\%}}\right) $$
link_by_source_by_property traversal_paths=255 1,resolve_depths=inherit:1;values:255;properties:255;links:127;link_dests:126;type:true $$73.9 \mathrm{ms} \pm 503 \mathrm{μs}\left({\color{gray}0.794 \mathrm{\%}}\right) $$
link_by_source_by_property traversal_paths=2 1,resolve_depths=inherit:0;values:0;properties:0;links:0;link_dests:0;type:false $$53.1 \mathrm{ms} \pm 425 \mathrm{μs}\left({\color{gray}4.08 \mathrm{\%}}\right) $$
link_by_source_by_property traversal_paths=2 1,resolve_depths=inherit:0;values:0;properties:0;links:1;link_dests:0;type:true $$61.7 \mathrm{ms} \pm 475 \mathrm{μs}\left({\color{gray}3.44 \mathrm{\%}}\right) $$
link_by_source_by_property traversal_paths=2 1,resolve_depths=inherit:0;values:0;properties:2;links:1;link_dests:0;type:true $$63.1 \mathrm{ms} \pm 399 \mathrm{μs}\left({\color{gray}1.56 \mathrm{\%}}\right) $$
link_by_source_by_property traversal_paths=2 1,resolve_depths=inherit:0;values:2;properties:2;links:1;link_dests:0;type:true $$63.2 \mathrm{ms} \pm 585 \mathrm{μs}\left({\color{gray}2.12 \mathrm{\%}}\right) $$

scenarios

Function Value Mean Flame graphs
full_test query-limited $$108 \mathrm{ms} \pm 847 \mathrm{μs}\left({\color{lightgreen}-5.564 \mathrm{\%}}\right) $$ Flame Graph
full_test query-unlimited $$120 \mathrm{ms} \pm 701 \mathrm{μs}\left({\color{lightgreen}-5.090 \mathrm{\%}}\right) $$ Flame Graph
linked_queries query-limited $$23.2 \mathrm{ms} \pm 190 \mathrm{μs}\left({\color{red}25.5 \mathrm{\%}}\right) $$ Flame Graph
linked_queries query-unlimited $$519 \mathrm{ms} \pm 1.28 \mathrm{ms}\left({\color{gray}-1.042 \mathrm{\%}}\right) $$ Flame Graph

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/apps > hash* Affects HASH (a `hash-*` app) area/apps > hash-graph area/apps area/deps Relates to third-party dependencies (area) area/libs Relates to first-party libraries/crates/packages (area) area/tests New or updated tests type/eng > backend Owned by the @backend team

Development

Successfully merging this pull request may close these issues.

3 participants