diff --git a/pages/memgraph-zero/memgql/_meta.ts b/pages/memgraph-zero/memgql/_meta.ts index ed520042c..287f964f9 100644 --- a/pages/memgraph-zero/memgql/_meta.ts +++ b/pages/memgraph-zero/memgql/_meta.ts @@ -3,6 +3,7 @@ export default { "use-cases": "Use Cases", "features": "Features", "connect": "Connect", + "schema-file": "Schema File", "multiple-graphs": "Multiple Graphs", "complete": "Docker Compose", "reference": "Reference", diff --git a/pages/memgraph-zero/memgql/changelog.mdx b/pages/memgraph-zero/memgql/changelog.mdx index b336ba81e..aa6a0523a 100644 --- a/pages/memgraph-zero/memgql/changelog.mdx +++ b/pages/memgraph-zero/memgql/changelog.mdx @@ -5,6 +5,37 @@ description: MemGQL release notes # MemGQL Changelog +## Unreleased + +### ⚠️ Breaking changes + +- **New graph model: connectors + schema graphs.** A **connector** is now a pure + connection and a **graph** is a mapping (`vertices` + `edges`) laid over one or + more connectors. Register a connection with `ADD CONNECTOR`, then define a graph + with `CREATE GRAPH FROM ''` / `FROM FILE ''`, or boot the + whole catalog from a single [`--schema`](/memgraph-zero/memgql/schema-file) file. +- **New mapping format.** Mappings use `vertices` / `edges` with + `mappedTableSource` (relational) or `mappedGraphSource` (native), `metaFields` + (`id` / `from` / `to`), and `attributes`. The legacy `nodes` / `edges` format + (`id_column`, `source_label`, `rel_type`) is no longer accepted; loading it + raises an actionable error pointing at the new format. +- **Removed statements.** `ADD GRAPH … ON CONNECTOR`, `UNADD GRAPH`, + `ALTER GRAPH SET CONNECTOR / GRAPH / MAPPING`, `ADD MAPPING` / `DROP MAPPING`, + and the connection-scoped `CONNECT … AS`, `DISCONNECT`, `USE CONNECTION`, and + `SET DEFAULT CONNECTION` are gone. Register connectors + graphs instead and + query by graph name (`USE `) or let routing pick the graph. + +### ✨ New features & Improvements + +- **Query graphs without `USE`.** [Schema-based routing](/memgraph-zero/memgql/multiple-graphs#routing) + infers the target graph from the labels, relationship types, and declared + properties a query mentions; ambiguity is an actionable error. Explicit + `USE ` remains the override. +- **Caching engages under routing.** A cache-enabled graph is populated + and served whether a part is pinned with `USE` or routed by label. Re-issuing + `ALTER GRAPH … SET CACHE` now drops stale fragments, so re-pointing the cache + can't serve a stale result. + ## MemGQL v0.7.0 - July 5th, 2026 ### ⚠️ Behavior changes @@ -58,7 +89,7 @@ description: MemGQL release notes RETURN e.name, i.total; ``` Previously this join had to be spelled as `WHERE i.employee_id = e.id`. Works - USE-free (each part routes by its own labels) and with multiple keys + without `USE` (each part routes by its own labels) and with multiple keys (`{country: o.country, city: o.city}`). - **SQL Server connector.** New `sqlserver` connector type (aliases: `mssql`, `sql_server`, `sql-server`) translates GQL to T-SQL. The driver is @@ -93,12 +124,12 @@ description: MemGQL release notes ### ✨ New features & Improvements -- **Schema-based routing (USE-free queries).** In `multi` mode, queries no +- **Schema-based routing.** In `multi` mode, queries no longer need a `USE ` clause — the engine infers the backend from the query's schema signals (labels, rel-types, properties) against a unified schema index, built from mappings for SQL backends and live introspection for Cypher backends. Routing is strict: exactly one candidate routes, zero or multiple - hard-error with the fix; explicit `USE` always wins. USE-free federated + hard-error with the fix; explicit `USE` always wins. Routed federated `JOIN` and `UNION` work too, and writes route on a unique candidate. Two new statements expose the index: **`SHOW SCHEMA [FOR ]`** and **`REFRESH SCHEMA`**. Identifier matching is now **exact** engine-wide. **Still gated diff --git a/pages/memgraph-zero/memgql/complete.mdx b/pages/memgraph-zero/memgql/complete.mdx index ab81b97ec..19709c312 100644 --- a/pages/memgraph-zero/memgql/complete.mdx +++ b/pages/memgraph-zero/memgql/complete.mdx @@ -50,7 +50,7 @@ docker network create memgql-net MemGQL needs a graph mapping to translate graph patterns into SQL. Create these two files next to your `docker-compose.yml`: -**`pg_init.sql`** — creates the relational tables: +**`pg_init.sql`** creates the relational tables: ```sql CREATE TABLE users ( @@ -60,36 +60,52 @@ CREATE TABLE users ( ); CREATE TABLE friend_of ( + id SERIAL PRIMARY KEY, user_id INTEGER NOT NULL REFERENCES users(id), friend_id INTEGER NOT NULL REFERENCES users(id), - PRIMARY KEY (user_id, friend_id) + UNIQUE (user_id, friend_id) ); ``` -**`pg_mapping.json`** — tells MemGQL how tables map to graph labels: +**`pg_mapping.json`** is a graph mapping body (`vertices` + `edges`) that maps the +tables to graph labels on the `pg` connector: ```json { - "nodes": [ + "vertices": [ { "label": "User", - "table": "users", - "id_column": "id", - "properties": { - "name": "name", - "email": "email" - } + "mappedTableSource": { + "connector": "pg", + "table": "users", + "metaFields": { + "id": "id" + } + }, + "attributes": [ + { + "name": "name" + }, + { + "name": "email" + } + ] } ], "edges": [ { - "rel_type": "FRIEND_OF", - "table": "friend_of", - "id_column": "id", - "source_column": "user_id", - "target_column": "friend_id", - "source_label": "User", - "target_label": "User" + "label": "FRIEND_OF", + "from": "User", + "to": "User", + "mappedTableSource": { + "connector": "pg", + "table": "friend_of", + "metaFields": { + "id": "id", + "from": "user_id", + "to": "friend_id" + } + } } ] } @@ -136,10 +152,8 @@ services: - -c - | echo "ADD CONNECTOR mg TYPE memgraph URI 'memgraph:7687' GRAPH memgraph;" | mgconsole --host memgql --port 7688 && - echo "CONNECT mg AS mg_conn;" | mgconsole --host memgql --port 7688 && - echo "ADD MAPPING pg_social FROM '/mappings/pg_mapping.json';" | mgconsole --host memgql --port 7688 && - echo "ADD CONNECTOR pg TYPE postgres URI 'host=postgres user=postgres password=postgres dbname=postgres' MAPPING pg_social;" | mgconsole --host memgql --port 7688 && - echo "CONNECT pg AS pg_conn;" | mgconsole --host memgql --port 7688 && + echo "ADD CONNECTOR pg TYPE postgres URI 'host=postgres user=postgres password=postgres dbname=postgres';" | mgconsole --host memgql --port 7688 && + echo "CREATE GRAPH social FROM FILE '/mappings/pg_mapping.json';" | mgconsole --host memgql --port 7688 && echo "MemGQL bootstrap complete." depends_on: memgql: @@ -248,7 +262,7 @@ docker compose ps You should see all eight services running: `memgraph`, `memgql`, `memgql-init`, `lab`, `postgres`, `memgql-mcp`, and `structured2graph`. The `memgql-init` -container registers both connectors and exits — this is expected. +container registers both connectors and exits; this is expected. ## 5. Connect with mgconsole @@ -256,13 +270,21 @@ container registers both connectors and exits — this is expected. mgconsole --port 7688 ``` -Both the Memgraph and PostgreSQL connectors are already registered and -connected by the `memgql-init` service. You can start querying right away. +The `mg` and `pg` connectors and the `social` graph (over PostgreSQL) are +already registered by the `memgql-init` service. We add the native Memgraph +graph below, then query each by its labels: routing picks the graph, so no +`USE` is required. ### Seed Memgraph with developers +Register a native graph over the Memgraph connector, then insert developers (the +`Developer` label routes to `dev`): + ```gql -SET DEFAULT CONNECTION mg_conn; +CREATE GRAPH dev FROM '{ + "vertices": [ { "label": "Developer", "mappedGraphSource": { "connector": "mg" } } ], + "edges": [ { "label": "FOLLOWS", "from": "Developer", "to": "Developer", "mappedGraphSource": { "connector": "mg" } } ] +}'; ``` ```gql @@ -289,11 +311,7 @@ MATCH (a:Developer)-[:FOLLOWS]->(b:Developer) RETURN a.name, b.name; ### Seed PostgreSQL with users -Switch to the PostgreSQL connection and insert users: - -```gql -SET DEFAULT CONNECTION pg_conn; -``` +Insert users; the `User` label routes to the `social` graph (PostgreSQL): ```gql INSERT (alice:User {name: "Alice", email: "alice@example.com"}); @@ -321,25 +339,20 @@ MATCH (u:User)-[:FRIEND_OF]->(f:User) RETURN u.name, f.name; ### Query across both backends -Each connection is queried independently — MemGQL routes the query to -the right backend based on the `USE CONNECTION` clause: - ```gql -USE CONNECTION mg_conn - MATCH (d:Developer)-[:FOLLOWS]->(f:Developer) +MATCH (d:Developer)-[:FOLLOWS]->(f:Developer) RETURN d.name AS source, "FOLLOWS" AS rel, f.name AS target; ``` ```gql -USE CONNECTION pg_conn - MATCH (u:User)-[:FRIEND_OF]->(f:User) +MATCH (u:User)-[:FRIEND_OF]->(f:User) RETURN u.name AS source, "FRIEND_OF" AS rel, f.name AS target; ``` ## 6. Open Memgraph Lab Open [http://localhost:3000](http://localhost:3000) in a browser. -Lab is pre-configured to connect to MemGQL on port 7688 — click +Lab is pre-configured to connect to MemGQL on port 7688; click **Quick Connect** and start exploring visually. ## 7. Create mappings with structured2graph diff --git a/pages/memgraph-zero/memgql/connect.mdx b/pages/memgraph-zero/memgql/connect.mdx index c4f144bcb..c4da52ddd 100644 --- a/pages/memgraph-zero/memgql/connect.mdx +++ b/pages/memgraph-zero/memgql/connect.mdx @@ -17,4 +17,4 @@ description: MemGQL connector details and configuration. ## Multi-Connector -Connect to multiple backends simultaneously. See the [Quick Start](/memgraph-zero/memgql/quick-start#multi-connection-mode) for details. +Connect to multiple backends simultaneously. See the [Quick Start](/memgraph-zero/memgql/quick-start#connecting-your-data-with-a-schema-file) for details. diff --git a/pages/memgraph-zero/memgql/connect/duckdb.mdx b/pages/memgraph-zero/memgql/connect/duckdb.mdx index cf23b4b2d..79d5b7d5f 100644 --- a/pages/memgraph-zero/memgql/connect/duckdb.mdx +++ b/pages/memgraph-zero/memgql/connect/duckdb.mdx @@ -90,7 +90,7 @@ For environment variables, see [Reference](../reference.mdx#duckdb-duckdb). | Typed edge `(a)-[r:R]->(b)` | ✓ | | Untyped edge `()-[]->(b)` | ✗ | | `UNION` / `UNION ALL` / `UNION DISTINCT` | ✓ | -| Quantified path `(){m,n}` — bounded | ✓ | +| Quantified path `(){m,n}` (bounded) | ✓ | | Whole-node `RETURN n` / whole-rel `RETURN r` | ✓ | | Map projections `RETURN n {.a, .b}` | ✓ | | `IN` list membership `WHERE x IN [...]` | ✓ | @@ -106,11 +106,11 @@ For environment variables, see [Reference](../reference.mdx#duckdb-duckdb). | `SET` / `REMOVE` (property update / delete) | ✗ | | `DETACH DELETE` | ✗ | | `INTERSECT` / `EXCEPT` | ✗ | -| Quantified path `(){m,}` — unbounded | ✗ | +| Quantified path `(){m,}` (unbounded) | ✗ | | Shortest-path (`ALL SHORTEST` / `SHORTEST k`) | ✗ | ## Known limitations - **Unbounded variable-length paths** (`()-[*]->()`) are rejected. Bound the depth (`{1,5}`) or run the query against a Cypher backend. - **`FOR x IN [...]` (UNWIND-style)** is rejected with an actionable error. Cypher backends (Memgraph, Neo4j) handle this syntax natively. -- **Path variables on variable-length patterns** — `MATCH p = (a){1,3}(b) RETURN p` is not supported. Drop the `p =` binding (or query a Cypher backend) and `RETURN` the individual nodes / edges instead. +- **Path variables on variable-length patterns**: `MATCH p = (a){1,3}(b) RETURN p` is not supported. Drop the `p =` binding (or query a Cypher backend) and `RETURN` the individual nodes / edges instead. diff --git a/pages/memgraph-zero/memgql/connect/iceberg.mdx b/pages/memgraph-zero/memgql/connect/iceberg.mdx index e57cbb1e7..91f537d81 100644 --- a/pages/memgraph-zero/memgql/connect/iceberg.mdx +++ b/pages/memgraph-zero/memgql/connect/iceberg.mdx @@ -13,7 +13,7 @@ Iceberg tables, and both fully qualify table references as | Connector | `CONNECTOR_TYPE` | Execution | Writes | |------------------|------------------|---------------------------------------------------------------------------|--------| -| Direct (native) | `iceberg-direct` | Native, in-process — reads Iceberg as Arrow directly from object storage | ❌ read-only | +| Direct (native) | `iceberg-direct` | Native, in-process; reads Iceberg as Arrow directly from object storage | ❌ read-only | | Trino | `iceberg` | GQL → Trino-dialect SQL, executed by a Trino engine | ✓ | **Which one should you use?** @@ -28,7 +28,7 @@ Iceberg tables, and both fully qualify table references as fully contained in one Iceberg catalog**, or when you need writes. Trino executes joins, pruning, and aggregation server-side. -Both connectors accept the same mapping file unchanged — a mapping written for +Both connectors accept the same mapping file unchanged; a mapping written for one works against the other. --- @@ -41,7 +41,7 @@ in-process** graph execution on Iceberg. It uses the [`iceberg-catalog-rest`](https://crates.io/crates/iceberg-catalog-rest) Rust crates to talk to the Iceberg REST Catalog and read table data as Apache Arrow record batches directly from object storage. There is **no Trino** and no SQL -string — MemGQL owns the entire execution loop. +string; MemGQL owns the entire execution loop. It is **read-only**: `INSERT`, `DELETE`, `SET`, and `REMOVE` return an explicit "unsupported" error. @@ -293,4 +293,4 @@ aggregation run **in-process** over Arrow batches. | Aggregation | ✓ (local) | ✓ | | Time-travel (snapshot) | ✓ | ✓ | | `INSERT (a {…})` | ❌ (read-only) | ✓ | -| `DELETE` (no alias — Trino requirement) | ❌ (read-only) | ✓ | +| `DELETE` (no alias; Trino requirement) | ❌ (read-only) | ✓ | diff --git a/pages/memgraph-zero/memgql/connect/mysql.mdx b/pages/memgraph-zero/memgql/connect/mysql.mdx index f232dc115..3431cf4c4 100644 --- a/pages/memgraph-zero/memgql/connect/mysql.mdx +++ b/pages/memgraph-zero/memgql/connect/mysql.mdx @@ -99,7 +99,7 @@ MATCH (a:Person)-[:KNOWS]->(b:Person) RETURN a.name, b.name; ## Dialect notes - Positional `?` parameter placeholders (no numbered `$1` form). -- No `RETURNING` on `INSERT` — use `last_insert_id()` if you need the +- No `RETURNING` on `INSERT`; use `last_insert_id()` if you need the generated row id. - Aliased single-table `DELETE` is emitted as the multi-table form `DELETE alias FROM table AS alias WHERE …` (MySQL 8.0+ requirement). @@ -121,7 +121,7 @@ For environment variables, see [Reference](../reference.mdx#mysql-mysql). | Typed edge `(a)-[r:R]->(b)` | ✓ | | Untyped edge `()-[]->(b)` | ✗ | | `UNION` / `UNION ALL` / `UNION DISTINCT` | ✓ | -| Quantified path `(){m,n}` — bounded | ✓ | +| Quantified path `(){m,n}` (bounded) | ✓ | | Whole-node `RETURN n` / whole-rel `RETURN r` | ✓ | | Map projections `RETURN n {.a, .b}` | ✓ | | `IN` list membership `WHERE x IN [...]` | ✓ | @@ -137,11 +137,11 @@ For environment variables, see [Reference](../reference.mdx#mysql-mysql). | `SET` / `REMOVE` (property update / delete) | ✗ | | `DETACH DELETE` | ✗ | | `INTERSECT` / `EXCEPT` | ✗ | -| Quantified path `(){m,}` — unbounded | ✗ | +| Quantified path `(){m,}` (unbounded) | ✗ | | Shortest-path (`ALL SHORTEST` / `SHORTEST k`) | ✗ | ## Known limitations - **Unbounded variable-length paths** (`()-[*]->()`) are rejected. Bound the depth (`{1,5}`) or run the query against a Cypher backend. - **`FOR x IN [...]` (UNWIND-style)** is rejected with an actionable error. Cypher backends (Memgraph, Neo4j) handle this syntax natively. -- **Path variables on variable-length patterns** — `MATCH p = (a){1,3}(b) RETURN p` is not supported. Drop the `p =` binding (or query a Cypher backend) and `RETURN` the individual nodes / edges instead. +- **Path variables on variable-length patterns**: `MATCH p = (a){1,3}(b) RETURN p` is not supported. Drop the `p =` binding (or query a Cypher backend) and `RETURN` the individual nodes / edges instead. diff --git a/pages/memgraph-zero/memgql/connect/oracle.mdx b/pages/memgraph-zero/memgql/connect/oracle.mdx index 94a832970..dce2bcb88 100644 --- a/pages/memgraph-zero/memgql/connect/oracle.mdx +++ b/pages/memgraph-zero/memgql/connect/oracle.mdx @@ -14,7 +14,7 @@ relational tables. The driver is [oracle-rs](https://crates.io/crates/oracle-rs), a **pure-Rust** implementation of Oracle's TNS wire protocol, pooled via [deadpool-oracle](https://crates.io/crates/deadpool-oracle). No OCI / ODPI-C / -Instant Client is required at build or runtime — the `memgraph/memgql:latest` +Instant Client is required at build or runtime: the `memgraph/memgql:latest` Docker image ships only the bolt server binary plus TLS roots, and `cargo run` works on macOS, Linux, and Windows without any extra system packages. @@ -122,14 +122,14 @@ MATCH (a:Person)-[:KNOWS]->(b:Person) RETURN a.name, b.name; - Positional bind variables use Oracle's native `:1`, `:2`, … form. - `LIMIT n OFFSET m` is rendered as the SQL-standard `OFFSET m ROWS FETCH NEXT n ROWS ONLY` (Oracle 12c+). -- Aliased single-table `DELETE` is `DELETE FROM table alias WHERE alias.col …` - — Oracle accepts table aliases but does not allow the `AS` keyword in this +- Aliased single-table `DELETE` is `DELETE FROM table alias WHERE alias.col …`. + Oracle accepts table aliases but does not allow the `AS` keyword in this position. - No `INSERT … RETURNING *`. Oracle supports `RETURNING INTO :out`, but this is bind-variable based and the planner stays bind-agnostic; the executor uses it only when the caller needs the auto-generated key. - Identifiers from the mapping file are emitted unquoted, so Oracle's - default UPPERCASE folding applies — match your mapping `table` / `id_column` + default UPPERCASE folding applies; match your mapping's `table` and column spellings to your physical schema accordingly. ## Multi-connection mode @@ -138,10 +138,9 @@ Inside `CONNECTOR_TYPE=multi`, register Oracle the same way as any other backend: ```gql -ADD MAPPING social FROM '/data/mapping.json'; -ADD CONNECTOR ora TYPE oracle URI 'oracle://system:oracle@oracle-dev:1521/FREEPDB1' MAPPING social; -CONNECT ora AS ora_conn; -USE CONNECTION ora_conn MATCH (n:Person) RETURN n.name LIMIT 5; +ADD CONNECTOR ora TYPE oracle URI 'oracle://system:oracle@oracle-dev:1521/FREEPDB1'; +CREATE GRAPH social FROM FILE '/data/mapping.json'; +MATCH (n:Person) RETURN n.name LIMIT 5; ``` For environment variables, see [Reference](../reference.mdx#oracle-oracle). diff --git a/pages/memgraph-zero/memgql/connect/postgres.mdx b/pages/memgraph-zero/memgql/connect/postgres.mdx index 8af24ef37..bc6b8e4dc 100644 --- a/pages/memgraph-zero/memgql/connect/postgres.mdx +++ b/pages/memgraph-zero/memgql/connect/postgres.mdx @@ -105,7 +105,7 @@ For environment variables, see [Reference](../reference.mdx#postgresql-postgres) | Typed edge `(a)-[r:R]->(b)` | ✓ | | Untyped edge `()-[]->(b)` | ✗ | | `UNION` / `UNION ALL` / `UNION DISTINCT` | ✓ | -| Quantified path `(){m,n}` — bounded | ✓ | +| Quantified path `(){m,n}` (bounded) | ✓ | | Whole-node `RETURN n` / whole-rel `RETURN r` | ✓ | | Map projections `RETURN n {.a, .b}` | ✓ | | `IN` list membership `WHERE x IN [...]` | ✓ | @@ -121,11 +121,11 @@ For environment variables, see [Reference](../reference.mdx#postgresql-postgres) | `SET` / `REMOVE` (property update / delete) | ✗ | | `DETACH DELETE` | ✗ | | `INTERSECT` / `EXCEPT` | ✗ | -| Quantified path `(){m,}` — unbounded | ✗ | +| Quantified path `(){m,}` (unbounded) | ✗ | | Shortest-path (`ALL SHORTEST` / `SHORTEST k`) | ✗ | ## Known limitations - **Unbounded variable-length paths** (`()-[*]->()`) are rejected. Bound the depth (`{1,5}`) or run the query against a Cypher backend. - **`FOR x IN [...]` (UNWIND-style)** is rejected with an actionable error. Cypher backends (Memgraph, Neo4j) handle this syntax natively. -- **Path variables on variable-length patterns** — `MATCH p = (a){1,3}(b) RETURN p` is not supported. Drop the `p =` binding (or query a Cypher backend) and `RETURN` the individual nodes / edges instead. +- **Path variables on variable-length patterns**: `MATCH p = (a){1,3}(b) RETURN p` is not supported. Drop the `p =` binding (or query a Cypher backend) and `RETURN` the individual nodes / edges instead. diff --git a/pages/memgraph-zero/memgql/connect/sqlserver.mdx b/pages/memgraph-zero/memgql/connect/sqlserver.mdx index de7874ec9..1ade90643 100644 --- a/pages/memgraph-zero/memgql/connect/sqlserver.mdx +++ b/pages/memgraph-zero/memgql/connect/sqlserver.mdx @@ -8,7 +8,7 @@ description: Connect MemGQL to Microsoft SQL Server. The SQL Server connector (type `sqlserver`; `mssql`, `sql_server`, and `sql-server` are accepted aliases) translates GQL queries into T-SQL. It requires a [mapping file](/memgraph-zero/memgql/reference#mapping-schema) that -maps graph patterns to relational tables — the same format as every other SQL +maps graph patterns to relational tables, the same format as every other SQL backend. The driver is [tiberius](https://crates.io/crates/tiberius), a **pure-Rust** @@ -16,7 +16,7 @@ implementation of the TDS wire protocol. No ODBC driver or system packages are required at build or runtime. SQL Server is available in -[`multi` mode](/memgraph-zero/memgql/multiple-graphs) only — there is no +[`multi` mode](/memgraph-zero/memgql/multiple-graphs) only; there is no standalone `CONNECTOR_TYPE=sqlserver` environment mode yet. You register it at runtime with `ADD CONNECTOR … TYPE sqlserver`, as shown below. @@ -79,7 +79,7 @@ SQL ## 3. Write the mapping -Save as `mapping.json` — the standard +Save as `mapping.json`, the standard [mapping format](/memgraph-zero/memgql/reference#mapping-schema). Declare every property you plan to query: in `multi` mode the mapping is also the routing schema, and a property that isn't declared is a routing error, not a @@ -87,26 +87,70 @@ passthrough. ```json { - "nodes": [ + "vertices": [ { - "label": "Person", "table": "persons", "id_column": "id", - "properties": { "name": "name", "age": "age" } + "label": "Person", + "mappedTableSource": { + "connector": "ss", + "table": "persons", + "metaFields": { + "id": "id" + } + }, + "attributes": [ + { + "name": "name" + }, + { + "name": "age", + "type": "Int" + } + ] }, { - "label": "Company", "table": "companies", "id_column": "id", - "properties": { "name": "name" } + "label": "Company", + "mappedTableSource": { + "connector": "ss", + "table": "companies", + "metaFields": { + "id": "id" + } + }, + "attributes": [ + { + "name": "name" + } + ] } ], "edges": [ { - "rel_type": "KNOWS", "table": "knows", "id_column": "id", - "source_column": "from_id", "target_column": "to_id", - "source_label": "Person", "target_label": "Person" + "label": "KNOWS", + "from": "Person", + "to": "Person", + "mappedTableSource": { + "connector": "ss", + "table": "knows", + "metaFields": { + "id": "id", + "from": "from_id", + "to": "to_id" + } + } }, { - "rel_type": "WORKS_AT", "table": "works_at", "id_column": "id", - "source_column": "person_id", "target_column": "company_id", - "source_label": "Person", "target_label": "Company" + "label": "WORKS_AT", + "from": "Person", + "to": "Company", + "mappedTableSource": { + "connector": "ss", + "table": "works_at", + "metaFields": { + "id": "id", + "from": "person_id", + "to": "company_id" + } + } } ] } @@ -133,11 +177,10 @@ mgconsole --port 7688 ``` ```gql -ADD MAPPING social FROM '/data/mapping.json'; ADD CONNECTOR ss TYPE sqlserver - URI 'Server=sqlserver-dev,1433;Database=test;User Id=sa;Password=Memgraph!2024;TrustServerCertificate=true' - MAPPING social; -CONNECT ss AS ss_conn; + URI 'Server=sqlserver-dev,1433;Database=test;User Id=sa;Password=Memgraph!2024;TrustServerCertificate=true'; +CREATE GRAPH social FROM FILE '/data/mapping.json'; +MATCH (n:Person) RETURN n.name LIMIT 5; ``` The `URI` is an ADO-style connection string @@ -165,18 +208,18 @@ MATCH (a:Person)-[:KNOWS]->(b:Person) RETURN a.name, b.name; is synthesized because T-SQL requires one for `OFFSET`/`FETCH`. - Booleans are emitted as `BIT` (`1`/`0`). - `char_length()` / `length()` on a column translate to T-SQL `LEN()`. -- No `INSERT … RETURNING` — SQL Server's `OUTPUT` clause is used only when the +- No `INSERT … RETURNING`; SQL Server's `OUTPUT` clause is used only when the caller needs the auto-generated key. ## Known limitations -- **Variable-length paths** — quantified path patterns (`(){1,3}`) and trail +- **Variable-length paths**: quantified path patterns (`(){1,3}`) and trail semantics are not available on SQL Server yet and return an actionable error. Use a bounded fixed-hop chain or a Cypher backend. -- **`collect()`** is not available — there is no portable array aggregate in +- **`collect()`** is not available; there is no portable array aggregate in the T-SQL surface MemGQL emits yet; the query fails with the backend's error. - **Map projections** (`RETURN n {.a, .b}`) currently return `NULL` instead of - a map (no error is raised) — don't rely on them on this backend. + a map (no error is raised); don't rely on them on this backend. - **`INSERT … RETURN`** executes the insert but returns the affected-row count instead of the projected values. diff --git a/pages/memgraph-zero/memgql/features.mdx b/pages/memgraph-zero/memgql/features.mdx index e079ae6c1..5a211e687 100644 --- a/pages/memgraph-zero/memgql/features.mdx +++ b/pages/memgraph-zero/memgql/features.mdx @@ -26,7 +26,7 @@ description: MemGQL Community and Enterprise feature comparison. | Max connectors | 2 | Unlimited | | Max simultaneous connections | 2 | Unlimited | | [Cross-backend joins & composite queries](/memgraph-zero/memgql/multiple-graphs) | Yes | Yes | -| [Schema-based routing (USE-free queries)](/memgraph-zero/memgql/multiple-graphs#use-free-routing) | Yes | Yes | +| [Schema-based query routing](/memgraph-zero/memgql/multiple-graphs#routing) | Yes | Yes | | [Query-driven graph cache (Memgraph)](/memgraph-zero/memgql/multiple-graphs#caching-a-graph-in-memgraph) | Yes | Yes | | **Agentic Capabilities** | | | | MCP Server | Yes | Yes | diff --git a/pages/memgraph-zero/memgql/multiple-graphs.mdx b/pages/memgraph-zero/memgql/multiple-graphs.mdx index 741a6a000..ed8822d1f 100644 --- a/pages/memgraph-zero/memgql/multiple-graphs.mdx +++ b/pages/memgraph-zero/memgql/multiple-graphs.mdx @@ -11,13 +11,13 @@ If you want a running stack to try these queries against, the [Docker Compose - ## Where the catalog DSL works -The catalog statements (`ADD CONNECTOR`, `ADD GRAPH`, `SHOW GRAPHS`, `SHOW CONNECTORS`, `DROP GRAPH`, `USE `, …) are available in **`CONNECTOR_TYPE=multi`** mode. Single-backend modes (`memgraph-gql`, `neo4j-gql`, `postgres`, `mysql`, `oracle`, `duckdb`, `clickhouse`, `iceberg`, `pinot`) connect to one backend configured via env vars and don't expose the catalog. +The catalog statements (`ADD CONNECTOR`, `CREATE GRAPH`, `SHOW GRAPHS`, `SHOW CONNECTORS`, `DROP GRAPH`, `USE `, …) are available in **`CONNECTOR_TYPE=multi`** mode. Single-backend modes (`memgraph-gql`, `neo4j-gql`, `postgres`, `mysql`, `oracle`, `duckdb`, `clickhouse`, `iceberg`, `pinot`) connect to one backend configured via env vars and don't expose the catalog. | Statement | `multi` | Cypher single-backend | SQL single-backend | |---|---|---|---| | `SHOW GRAPHS` / `SHOW CONNECTORS` / `SHOW MAPPINGS` | ✓ | ✗ | ✗ | | `SHOW SCHEMA` / `REFRESH SCHEMA` | ✓ | ✗ | ✗ | -| `ADD CONNECTOR` / `ADD GRAPH` / `CONNECT` | ✓ | ✗ | ✗ | +| `ADD CONNECTOR` / `CREATE GRAPH … FROM` / `PING` | ✓ | ✗ | ✗ | | `CREATE GRAPH ` | ✓ (catalog entry) | ✓ (forwarded as `CREATE DATABASE `) | ✗ | | `DROP GRAPH ` | ✓ | ✗ | ✗ | | `TRUNCATE ` | ✓ | ✓ (forwarded as `MATCH (n) DETACH DELETE n`) | ✗ | @@ -27,49 +27,50 @@ If you need graph management today, run MemGQL in `multi` mode. ## Graph Registration -Before querying across graphs, register them in the catalog using `ADD GRAPH` or `CREATE GRAPH`: +Register connectors first, then define graphs over them. A **connector** is a +connection; a **graph** is a mapping (`vertices` + `edges`) laid over one or more +connectors. ```gql --- Register existing graphs on different backends -ADD GRAPH social ON CONNECTOR neo4j_prod GRAPH neo4j; -ADD GRAPH events ON CONNECTOR clickhouse_logs READ ONLY; -ADD GRAPH warehouse ON CONNECTOR postgres_dw MAPPING company_mapping READ ONLY; -ADD GRAPH dev ON CONNECTOR memgraph_dev; - --- No-op if the name is already registered (re-runnable setup scripts) -ADD GRAPH IF NOT EXISTS social ON CONNECTOR neo4j_prod GRAPH neo4j; +-- 1. Register connectors (connections only) +ADD CONNECTOR neo4j_prod TYPE neo4j URI 'bolt://neo4j:7687' USER 'neo4j' PASSWORD 'secret' GRAPH neo4j; +ADD CONNECTOR clickhouse_logs TYPE clickhouse URI 'http://clickhouse:8123'; +ADD CONNECTOR postgres_dw TYPE postgres URI 'postgresql://pg:5432/dw'; + +-- 2. Define graphs over them (mapping body from a file, or inline JSON) +CREATE GRAPH social FROM FILE '/data/social.graph.json'; +CREATE GRAPH events FROM FILE '/data/events.graph.json'; +CREATE GRAPH warehouse FROM FILE '/data/warehouse.graph.json'; + +-- read-only access where you want it +ALTER GRAPH events SET READ ONLY; +ALTER GRAPH warehouse SET READ ONLY; ``` -Each graph entry stores: -- **Name**: Local catalog identifier -- **Connector**: Which connector hosts this graph -- **Remote graph**: Backend-specific graph/database name (optional) -- **Mapping**: Schema mapping for relational connectors (optional) -- **Access mode**: `READ WRITE` (default) or `READ ONLY` +Each graph carries a name, its per-connector mapping (`vertices` / `edges`; see +the [Schema File](/memgraph-zero/memgql/schema-file) page), and an access mode +(`READ WRITE` default, or `READ ONLY`). A single graph may span several +connectors. ## Single Graph Queries -Once registered, use graphs by name without specifying connectors: +Reference a label and the query routes to the graph that defines it, with no +connector or `USE` clause needed: ```gql --- Query the social graph (resolves to neo4j_prod) -USE social MATCH (p:Person) RETURN p.name; - --- Query the events graph (read-only ClickHouse) -USE events MATCH (e:Event) WHERE e.ts > '2025-01-01' RETURN e; - --- Query the warehouse with mapping translation -USE warehouse MATCH (c:Company) WHERE c.revenue > 1000000 RETURN c; +MATCH (p:Person) RETURN p.name; +MATCH (e:Event) WHERE e.ts > '2025-01-01' RETURN e; +MATCH (c:Company) WHERE c.revenue > 1000000 RETURN c; ``` -The engine resolves the graph name to its bound connector and executes the query on the appropriate backend. +The engine resolves each label to its owning graph and executes the query on the appropriate backend. -## USE-free routing +## Routing You don't always have to name the graph. MemGQL maintains a **unified schema -index** — the labels, relationship types, and properties every registered -source defines — and uses it to infer which backend a query belongs to from the -query itself. A query with no `USE` clause routes automatically when its schema +index** (the labels, relationship types, and properties every registered source +defines) and uses it to infer which backend a query belongs to from the query +itself. A query with no `USE` clause routes automatically when its schema signals point to exactly one source. ```gql @@ -83,9 +84,9 @@ MATCH (me:Person {id: 1})-[:FRIEND_OF]->(f:Person) RETURN f.id; The schema index is built from two sources: - **SQL-family connectors** (PostgreSQL, MySQL, Oracle, DuckDB, ClickHouse, - Iceberg, Pinot) — taken from the registered **mapping** (labels, rel-types, + Iceberg, Pinot): taken from the registered **mapping** (labels, rel-types, and properties are known exactly). -- **Cypher-family connectors** (Memgraph, Neo4j) — **introspected at `CONNECT` +- **Cypher-family connectors** (Memgraph, Neo4j): **introspected at `CONNECT` time** and cached. Memgraph uses `SHOW SCHEMA INFO` (the server must run with `--schema-info-enabled`); Neo4j uses `db.schema.*`, falling back to a labels-and-rel-types-only view. When property names can't be introspected, a @@ -103,22 +104,19 @@ Routing is strict and deterministic: Disambiguate by referencing a property or relationship type that exists in only one of them, or add an explicit `USE`. - **Explicit `USE` always wins** and bypasses inference entirely. -- **Session defaults never tie-break.** A sticky `default_connection` (from - `SET DEFAULT CONNECTION`) does not resolve an otherwise-ambiguous query. -- **No-signal queries** like `MATCH (n) RETURN n` (no label, rel-type, or - property to route by) fall back to today's default-connection behavior - unchanged. +- **No-signal queries** (`MATCH (n) RETURN n` with no label, relationship + type, or property to route by) have nothing to infer from; pin them with an + explicit `USE `. Identifier matching is **exact**: `:person` does not match a mapping (or introspected schema) that declares `Person`. Routing, translation, and the backends all agree on the same casing. -### USE-free federated queries +### Federated queries Routing happens per query part, so multi-backend joins and composites work -without any `USE` clauses — each part (or branch) routes independently and the -existing [cross-backend hash join](#focused-multi-graph-queries) takes over at -the boundary: +without any `USE` clauses; each part (or branch) routes independently and the +[cross-backend join](#focused-multi-graph-queries) takes over at the boundary: ```gql -- Federated JOIN with zero USE clauses: the FRIEND_OF part routes to social, @@ -140,7 +138,7 @@ follow-up read routes without needing a `REFRESH SCHEMA`. ### What still needs an explicit `USE` - A graph bound to a **non-default remote database** (Memgraph multi-tenancy) is - fenced to explicit `USE` — per-tenant introspection isn't wired up yet. + fenced to explicit `USE`; per-tenant introspection isn't wired up yet. - A **single `MATCH` pattern that spans two backends** errors with guidance to split it into one `MATCH` clause per graph; auto-splitting one pattern is out of scope. @@ -149,7 +147,7 @@ follow-up read routes without needing a `REFRESH SCHEMA`. ## Schema discovery -`SHOW SCHEMA` is the discoverability surface for USE-free routing — it's the +`SHOW SCHEMA` is the discoverability surface for routing; it's the unified index the router consults, and what routing errors point you at. It's especially useful for agents that need to learn what's queryable without hardcoded knowledge. @@ -184,7 +182,7 @@ SHOW SCHEMA FOR social; ``` `REFRESH SCHEMA` re-introspects every live Cypher connection (Memgraph / Neo4j) -and updates the cache. Introspection already runs eagerly at `CONNECT` — use +and updates the cache. Introspection already runs eagerly at `CONNECT`; use this after the underlying schema changes: ```gql @@ -200,7 +198,7 @@ REFRESH SCHEMA; +-------------+-------------+--------------------------------------------+ ``` -SQL-family connections report `mapping-backed (nothing to introspect)` — their +SQL-family connections report `mapping-backed (nothing to introspect)`; their schema comes from the mapping, not from a live query. If introspection fails (for example, a Memgraph started without `--schema-info-enabled`), the status explains why; the connection stays reachable via explicit `USE`. @@ -238,8 +236,8 @@ RETURN friend.email AS email, e.item AS item, e.amount AS amount; ### Piping: fetching only what joins -Since v0.7.0, when one side of a two-backend query is selective — it carries a -literal property filter, a `LIMIT`, or a bounded `CALL` — MemGQL runs that side +Since v0.7.0, when one side of a two-backend query is selective (it carries a +literal property filter, a `LIMIT`, or a bounded `CALL`), MemGQL runs that side first and uses its join keys to fetch only the matching rows from the other backend, instead of pulling the whole table: @@ -251,9 +249,9 @@ MATCH (p:Profile {uid: u.uid}) RETURN p.name, u.rank ORDER BY u.rank DESC; ``` -Piping is an optimization, not a semantic change — results are identical with +Piping is an optimization, not a semantic change; results are identical with or without it. It engages for backends registered as catalog graphs -(`ADD GRAPH`); when it doesn't apply, MemGQL pulls both sides and joins +(via `CREATE GRAPH`); when it doesn't apply, MemGQL pulls both sides and joins locally. If the selective side matches nothing, the other backend is never queried. @@ -366,56 +364,52 @@ SHOW GRAPH social; ``` `SHOW GRAPHS` lists how graphs are *registered* (connector, mapping, access -mode). To see what each one actually *defines* — the labels, relationship types, -and properties used for routing — use [`SHOW SCHEMA`](#schema-discovery). +mode). To see what each one actually *defines* (the labels, relationship types, +and properties used for routing), use [`SHOW SCHEMA`](#schema-discovery). ## Graph Lifecycle Management -### Creating Graphs +### Creating graphs -`CREATE GRAPH` both registers the graph and creates it on the remote backend: +`CREATE GRAPH … FROM` registers a graph from a mapping body (inline JSON or a +file) and auto-connects the connectors it references: ```gql --- Create an empty graph on a specific backend -CREATE GRAPH analytics ON CONNECTOR memgraph_dev; - --- Create with a GQL schema -CREATE GRAPH typed_graph { - NODE Person ({name STRING, age INT}), - EDGE KNOWS ()-[]->() -} ON CONNECTOR neo4j_prod; +-- from a file +CREATE GRAPH analytics FROM FILE '/data/analytics.graph.json'; + +-- or inline +CREATE GRAPH social FROM '{ + "vertices": [ { "label": "User", "mappedGraphSource": { "connector": "neo4j_prod" } } ], + "edges": [ { "label": "FOLLOWS", "from": "User", "to": "User", "mappedGraphSource": { "connector": "neo4j_prod" } } ] +}'; ``` -### Modifying Graphs +See the [Schema File](/memgraph-zero/memgql/schema-file) page for the mapping +body format. + +### Modifying graphs ```gql -- Change access mode ALTER GRAPH social SET READ ONLY; ALTER GRAPH social SET READ WRITE; - --- Rebind to a different connector (e.g., failover) -ALTER GRAPH social SET CONNECTOR neo4j_staging; - --- Change the remote graph name -ALTER GRAPH social SET GRAPH production_db; - --- Attach or change a mapping -ALTER GRAPH knowledge SET MAPPING updated_mapping; -ALTER GRAPH knowledge REMOVE MAPPING; ``` -### Removing Graphs +A graph's mapping is fixed at creation. To change it, drop and re-create the +graph with the new body (`DROP GRAPH`, then `CREATE GRAPH … FROM`). -```gql --- Remove from catalog only (remote graph is untouched) -UNADD GRAPH social; -UNADD GRAPH IF EXISTS social; +### Removing graphs --- Remove from catalog AND drop on backend -DROP GRAPH analytics; -DROP GRAPH IF EXISTS analytics; +```gql +DROP GRAPH social; +DROP GRAPH IF EXISTS social; ``` +`DROP GRAPH` removes the graph from the catalog; the backend data is untouched. +Connectors are removed separately with `DROP CONNECTOR` (refused while a graph +still references the connector). + ## Caching a graph in Memgraph Since v0.7.0, a graph can be **cached in a Memgraph instance**. This is aimed at @@ -426,32 +420,38 @@ itself (populate-on-read), so only the subgraph your queries actually touch lands in Memgraph. Register a Memgraph connector to hold the cached data, then mark a graph -cache-enabled with a `CACHE CONNECTOR` clause: +cache-enabled with `ALTER GRAPH … SET CACHE`: ```gql -- A Memgraph instance (storage mode IN_MEMORY_ANALYTICAL) holds the cache ADD CONNECTOR mg TYPE memgraph URI 'memgraph:7687'; --- Cache the warehouse graph in `mg`, with optional bounds -ADD GRAPH events ON CONNECTOR ice MAPPING warehouse READ ONLY - CACHE CONNECTOR mg TTL 3600 MAX_BYTES 8G; +-- A warehouse graph that reads from Iceberg (connectors registered earlier) +CREATE GRAPH events FROM FILE '/data/events.graph.json'; --- Or enable/disable caching on an already-registered graph +-- Enable caching in `mg`, with optional bounds ALTER GRAPH events SET CACHE CONNECTOR mg TTL 3600 MAX_BYTES 8G; + +-- Disable caching (and drop the cached fragments) ALTER GRAPH events REMOVE CACHE; ``` +The cache connector must be a Memgraph-type connector. Re-issuing `SET CACHE` +drops any fragments already recorded, so re-pointing the cache never serves a +stale result. + `TTL` is in seconds; `MAX_BYTES` accepts a plain byte count or a `K`/`M`/`G`/`T` binary suffix (`8G` = 8 × 1024³ bytes). -How it behaves per `USE` part: +The cache engages per routed part, whether the part is pinned with `USE` or +routed by its labels: -- **First read (cache MISS)** — the query runs on the source and the engine +- **First read (cache MISS)**: the query runs on the source and the engine *tees* the scanned labels and edge types into the Memgraph cache, recording which scopes it cached. -- **Later covered read (cache HIT)** — a query whose scans are covered by +- **Later covered read (cache HIT)**: a query whose scans are covered by existing cached scopes is served entirely from Memgraph, with no source I/O. -- **Capability upgrade** — because cached data is a real graph in Memgraph, +- **Capability upgrade**: because cached data is a real graph in Memgraph, queries over cached scopes can use Memgraph's full capability profile (variable-length paths, quantified patterns, graph algorithms) even when the source connector cannot express them: @@ -463,8 +463,9 @@ How it behaves per `USE` part: RETURN a.event_id, b.event_id; ``` -In a cross-connector (multi-`USE`) query, each cache-enabled part is served -from its cache independently while the other parts run on their own sources. +In a cross-backend query (split by explicit `USE`, or routed by label), each +cache-enabled part is served from its cache independently while the other parts +run on their own sources. Inspect what is cached with `SHOW GRAPH CACHES` (returns each cache-enabled graph, its cache connector, `TTL`, `MAX_BYTES`, and the number of cached @@ -484,7 +485,7 @@ and `[CACHE] HIT … serving from cache connector …`. 4. **Compatible column types**: All branches of a composite query must produce result sets with the same number of columns, in the same order, with compatible types. -5. **Routing is per part**: With [USE-free routing](#use-free-routing) each query part (and each composite branch) routes on its own schema signals. A part that matches zero or multiple sources is a hard error — add an explicit `USE` or reference a label/property unique to one source. +5. **Routing is per part**: With [routing](#routing) each query part (and each composite branch) routes on its own schema signals. A part that matches zero or multiple sources is a hard error; add an explicit `USE` or reference a label/property unique to one source. ## Complete Example @@ -496,10 +497,12 @@ ADD CONNECTOR neo4j_prod TYPE neo4j URI 'bolt://neo4j:7687' USER 'neo4j' PASSWOR ADD CONNECTOR pg_warehouse TYPE postgres URI 'postgresql://pg:5432/dw'; ADD CONNECTOR ch_logs TYPE clickhouse URI 'http://clickhouse:8123'; --- 2. Register graphs -ADD GRAPH social ON CONNECTOR neo4j_prod GRAPH neo4j; -ADD GRAPH warehouse ON CONNECTOR pg_warehouse MAPPING company_mapping READ ONLY; -ADD GRAPH events ON CONNECTOR ch_logs READ ONLY; +-- 2. Define graphs over the connectors (mapping bodies live in files) +CREATE GRAPH social FROM FILE '/data/social.graph.json'; +CREATE GRAPH warehouse FROM FILE '/data/warehouse.graph.json'; +CREATE GRAPH events FROM FILE '/data/events.graph.json'; +ALTER GRAPH warehouse SET READ ONLY; +ALTER GRAPH events SET READ ONLY; -- 3. Single-graph queries USE social MATCH (p:Person) RETURN p.name LIMIT 5; @@ -519,7 +522,7 @@ UNION USE warehouse MATCH (e:Employee) RETURN e.email AS email; -- 6. Cleanup -UNADD GRAPH social; -UNADD GRAPH warehouse; -UNADD GRAPH events; +DROP GRAPH social; +DROP GRAPH warehouse; +DROP GRAPH events; ``` diff --git a/pages/memgraph-zero/memgql/quick-start.mdx b/pages/memgraph-zero/memgql/quick-start.mdx index 7e03bb834..2bb2f8d3f 100644 --- a/pages/memgraph-zero/memgql/quick-start.mdx +++ b/pages/memgraph-zero/memgql/quick-start.mdx @@ -148,151 +148,226 @@ environment variables and connector-specific settings. ## Mapping File -Relational connectors (PostgreSQL, MySQL, Oracle, DuckDB, ClickHouse, Iceberg, and -Apache Pinot) and multi-connection mode require a JSON mapping file that defines -how graph patterns map to relational tables. Node labels map to tables and edge types map to association tables, -allowing GQL patterns like `MATCH (p:Person)-[:WORKS_AT]->(c:Company)` to be -translated into SQL JOINs. +A mapping describes how a relational backend's tables appear as graph labels and +relationship types: node labels over tables (`vertices`), edge types over +association tables (`edges`). The full field reference is on the +[Schema File](/memgraph-zero/memgql/schema-file) page. The example below is a +standalone mapping body, the shape `MAPPING_FILE` expects. Create a mapping file: ```bash cat > mapping.json << 'EOF' { - "nodes": [ + "vertices": [ { "label": "Person", - "table": "persons", - "id_column": "id", - "properties": { "name": "name", "age": "age" } + "mappedTableSource": { + "table": "persons", + "metaFields": { + "id": "id" + } + }, + "attributes": [ + { + "name": "name" + }, + { + "name": "age", + "type": "Int" + } + ] }, { "label": "Company", - "table": "companies", - "id_column": "id", - "properties": { "name": "name" } + "mappedTableSource": { + "table": "companies", + "metaFields": { + "id": "id" + } + }, + "attributes": [ + { + "name": "name" + } + ] } ], "edges": [ { - "rel_type": "KNOWS", - "table": "knows", - "id_column": "id", - "source_column": "from_id", - "target_column": "to_id", - "source_label": "Person", - "target_label": "Person" + "label": "KNOWS", + "from": "Person", + "to": "Person", + "mappedTableSource": { + "table": "knows", + "metaFields": { + "id": "id", + "from": "from_id", + "to": "to_id" + } + } }, { - "rel_type": "WORKS_AT", - "table": "works_at", - "id_column": "id", - "source_column": "person_id", - "target_column": "company_id", - "source_label": "Person", - "target_label": "Company" + "label": "WORKS_AT", + "from": "Person", + "to": "Company", + "mappedTableSource": { + "table": "works_at", + "metaFields": { + "id": "id", + "from": "person_id", + "to": "company_id" + } + } } ] } EOF ``` -Without `MAPPING_FILE`, a default Person/Company mapping (shown above) is used. +## Connecting your data with a schema file -## Multi-Connection Mode +Declare your backends and the graphs over them in one file, boot from it, and +query straight away, then add more whenever you need. -Multi-connection mode (`CONNECTOR_TYPE=multi`) lets you manage multiple backend -connectors and connections at runtime using MemGQL statements. You can add -connectors, establish connections, and route queries to specific backends — all -from within the same Bolt session. +### 1. Write a schema file -### 1. Start MemGQL in multi mode +A schema file declares the backends (`connectors`) and the graphs mapped over +them. Here is a minimal `schema.json` with one Memgraph connector and an +`engineering` graph over the data you seeded above: + +```json +{ + "connectors": [ + { + "name": "mg", + "type": "memgraph", + "connection": { + "uri": "memgraph-dev:7687" + } + } + ], + "graphs": [ + { + "name": "engineering", + "vertices": [ + { + "label": "Developer", + "mappedGraphSource": { + "connector": "mg" + } + }, + { + "label": "Language", + "mappedGraphSource": { + "connector": "mg" + } + } + ], + "edges": [ + { + "label": "WRITES", + "from": "Developer", + "to": "Language", + "mappedGraphSource": { + "connector": "mg" + } + } + ] + } + ] +} +``` + +See the [Schema File](/memgraph-zero/memgql/schema-file) page for every field. + +### 2. Boot from it + +Point MemGQL at the file. Every connector connects and every graph registers at +startup, with no further setup. This reuses the Memgraph you started above, so +stop the MemGQL container you started earlier (Ctrl-C) to free port 7688: ```bash -docker run --rm \ - --name memgql \ +docker run --rm -p 7688:7688 \ --network memgql-net \ - --stop-timeout 2 \ - -p 7688:7688 \ - --env CONNECTOR_TYPE=multi \ - --env BOLT_LISTEN_ADDR=0.0.0.0:7688 \ - -v ./mapping.json:/data/mapping.json \ - memgraph/memgql:latest + -e BOLT_LISTEN_ADDR=0.0.0.0:7688 \ + -v "$(pwd)/schema.json:/data/schema.json" \ + memgraph/memgql:latest --schema=/data/schema.json ``` -### 2. Connect and manage backends +### 3. Query ```bash mgconsole --port 7688 ``` -Add connectors and establish connections: +No `USE` clause is needed; the query routes to `engineering` by its labels. ```gql -ADD CONNECTOR mg TYPE memgraph URI 'memgraph-dev:7687' GRAPH memgraph; -CONNECT mg AS mg_conn; - -ADD CONNECTOR neo TYPE neo4j URI 'neo4j-dev:7687' GRAPH neo4j; -CONNECT neo AS neo_conn; +MATCH (d:Developer)-[:WRITES]->(l:Language) RETURN d.name, l.name LIMIT 5; ``` -### 3. Route queries to specific connections +### 4. Add more backends at runtime + +Register more connectors and graphs live, without a restart. Changes persist +back to `schema.json`. For example, with a PostgreSQL reachable on the same +network, add it and map a `store` graph over one of its tables: ```gql -USE CONNECTION mg_conn MATCH (n) RETURN n.name LIMIT 5; -USE CONNECTION neo_conn MATCH (n) RETURN n.name LIMIT 5; -``` +ADD CONNECTOR pg TYPE postgres URI 'postgresql://postgres:postgres@postgres-dev:5432/postgres'; -### 4. Introspection +CREATE GRAPH store FROM '{ + "vertices": [ + { + "label": "Customer", + "mappedTableSource": { + "connector": "pg", + "table": "customers", + "metaFields": { "id": "id" } + }, + "attributes": [ { "name": "name" } ] + } + ] +}'; -```gql -SHOW CONNECTORS; -SHOW CONNECTIONS; -SHOW MAPPINGS; -PING mg_conn; +MATCH (c:Customer) RETURN c.name LIMIT 5; ``` -### 5. Adding relational backends +`CREATE GRAPH … FROM` also accepts `FROM FILE ''` (the [mapping +file](#mapping-file) format). Pinot can additionally run as a single backend via +the `CONNECTION_TYPE` alias (`CONNECTION_TYPE=pinot PINOT_URL=…`). -Relational connectors (PostgreSQL, MySQL, Oracle, DuckDB, ClickHouse, Iceberg, and -Apache Pinot) require a mapping that defines how graph patterns map to tables: +### 5. Introspection ```gql -ADD MAPPING social FROM '/data/mapping.json'; -ADD CONNECTOR pg TYPE postgres USER 'postgres' PASSWORD 'postgres' DATABASE 'postgres' MAPPING social; -CONNECT pg AS pg_conn; -USE CONNECTION pg_conn MATCH (n:Person) RETURN n.name LIMIT 5; -``` - -Pinot can also run as a single backend with the alias requested by deployment -environments that use connection naming: - -```bash -CONNECTION_TYPE=pinot PINOT_URL=http://localhost:8099 cargo run --bin bolt_server +SHOW CONNECTORS; +SHOW GRAPHS; +SHOW MAPPINGS; +SHOW SCHEMA; +PING mg; ``` ### 6. Cleanup ```gql -DISCONNECT pg_conn; +DROP GRAPH store; DROP CONNECTOR pg; -DROP MAPPING social; ``` -### MemGQL Statements Reference - -| Statement | Description | -|----------------------------------------------------|---------------------------------------| -| `ADD MAPPING FROM ''` | Load a graph mapping from a JSON file | -| `DROP MAPPING ` | Remove a mapping | -| `ADD CONNECTOR TYPE [options...]` | Register a backend connector | -| `DROP CONNECTOR ` | Remove a connector | -| `CONNECT AS [DEFAULT]` | Establish a named connection | -| `DISCONNECT ` | Close a connection | -| `PING ` | Test a connection is alive | -| `USE CONNECTION ` | Route a query to a specific connection| -| `SET DEFAULT CONNECTOR ` | Set session default connector | -| `SET DEFAULT CONNECTION ` | Set session default connection | -| `SHOW CONNECTORS` | List registered connectors | -| `SHOW CONNECTIONS` | List active connections | -| `SHOW MAPPINGS` | List loaded mappings | +## MemGQL Statements Reference + +| Statement | Description | +|----------------------------------------------------------|------------------------------------------| +| `ADD CONNECTOR TYPE [options...]` | Register a backend connector (connection)| +| `DROP CONNECTOR ` | Remove a connector | +| `CREATE GRAPH FROM '' \| FROM FILE ''`| Define a graph (mapping) over connectors | +| `DROP GRAPH [IF EXISTS] ` | Remove a graph | +| `ALTER GRAPH SET READ ONLY \| READ WRITE` | Toggle a graph's access mode | +| `ALTER GRAPH SET CACHE CONNECTOR [...]` | Cache a graph in Memgraph | +| `USE ` | Route a query to a named graph | +| `PING ` | Test a connector is alive | +| `SHOW CONNECTORS` / `SHOW CONNECTIONS` | List connectors / live connections | +| `SHOW GRAPHS` / `SHOW GRAPH ` | List graphs / one graph's details | +| `SHOW MAPPINGS` | List per-graph mappings | +| `SHOW SCHEMA` / `REFRESH SCHEMA` | Routing index / re-introspect | +| `EXPORT SCHEMA [TO '']` | Export the catalog as JSON | diff --git a/pages/memgraph-zero/memgql/reference.mdx b/pages/memgraph-zero/memgql/reference.mdx index fe0479e96..467f904bc 100644 --- a/pages/memgraph-zero/memgql/reference.mdx +++ b/pages/memgraph-zero/memgql/reference.mdx @@ -27,8 +27,8 @@ supports. | Untyped edge `()-[]->(b)` (union over types) | ✓ | ✗ | | `UNION` / `UNION ALL` / `UNION DISTINCT` | ✓ | ✓ | | `INTERSECT` / `EXCEPT` | ✓ | ✓ | -| Quantified path `(){m,n}` — bounded | ✓ | ✓ | -| Quantified path `(){m,}` — unbounded | ✓ | ✗ | +| Quantified path `(){m,n}` (bounded) | ✓ | ✓ | +| Quantified path `(){m,}` (unbounded) | ✓ | ✗ | | Shortest-path (`ALL SHORTEST` / `ANY SHORTEST` / `SHORTEST k`) | ✓ | ✗ | | Whole-node `RETURN n` / whole-relationship `RETURN r` | ✓ | ✓ | | Map projections `RETURN n {.id, .title}` | ✓ | ✓ | @@ -59,10 +59,10 @@ supports. - **`FOR x IN [...]` (UNWIND-style) on SQL backends** returns an actionable error pointing users at running the query on a Cypher backend. The form is still accepted natively on Cypher backends. -- **`DETACH DELETE` on SQL backends** currently executes as a plain `DELETE` — +- **`DETACH DELETE` on SQL backends** currently executes as a plain `DELETE`; relationship rows are not detached. It deletes a node that has no relationship rows and otherwise surfaces the backend's constraint error; don't rely on it. -- **Path variables on variable-length patterns** — +- **Path variables on variable-length patterns**: `MATCH p = (a){1,3}(b) RETURN p` is not yet supported on SQL backends. Drop the `p =` binding (or query a Cypher backend) and `RETURN` the individual nodes / edges instead. @@ -70,44 +70,54 @@ supports. ## Graph Management Query Syntax ``` -ADD GRAPH [IF NOT EXISTS] ON CONNECTOR - [GRAPH ] - [MAPPING ] - [READ ONLY] - [CACHE CONNECTOR [TTL ] [MAX_BYTES [K|M|G|T]]]; - -CREATE GRAPH - [{ } | ANY] - ON CONNECTOR ; - -UNADD GRAPH [IF EXISTS] ; +-- Connectors (connections only) +ADD CONNECTOR TYPE + [URI ''] [PATH ''] + [USER ''] [PASSWORD ''] + [CATALOG ''] [SCHEMA ''] [GRAPH '']; +DROP CONNECTOR ; +PING ; + +-- Graphs (mappings over connectors) +CREATE GRAPH FROM ''; -- inline { "vertices": …, "edges": … } body +CREATE GRAPH FROM FILE ''; -- same body from a file DROP GRAPH [IF EXISTS] ; - ALTER GRAPH SET READ ONLY; ALTER GRAPH SET READ WRITE; -ALTER GRAPH SET CONNECTOR ; -ALTER GRAPH SET GRAPH ; -ALTER GRAPH SET MAPPING ; -ALTER GRAPH REMOVE MAPPING; + +-- Query-driven cache ALTER GRAPH SET CACHE CONNECTOR [TTL ] [MAX_BYTES [K|M|G|T]]; ALTER GRAPH REMOVE CACHE; - SHOW GRAPH CACHES; -- graph, cache_connector, ttl_secs, max_bytes, fragment count ``` -`ADD GRAPH IF NOT EXISTS` succeeds as a no-op when the name is already -registered — for re-runnable setup scripts; plain `ADD GRAPH` errors on a -duplicate name. +A connector is a **connection only**; it carries no graph shape. `GRAPH ` +selects the Cypher database on Memgraph / Neo4j (it is not a mapping). Re-adding +a connector replaces its config; `DROP CONNECTOR` is refused while a graph still +references it. -`CACHE CONNECTOR` makes a graph **cache-enabled**: its scans are populated into -a Memgraph cache connector on first read, and later covered queries are served -from that cache — see [Multiple Graphs → Caching](/memgraph-zero/memgql/multiple-graphs#caching-a-graph-in-memgraph). +`CREATE GRAPH … FROM` registers a graph from a `{ "vertices": …, "edges": … }` +body and auto-connects the connectors it references. The mapping format is +documented on the [Schema File](/memgraph-zero/memgql/schema-file) page; the same +body loads at boot via `--schema`. + +`SET CACHE CONNECTOR` makes a graph **cache-enabled**: touched labels and edge +types are copied into the Memgraph cache connector on first read, and later +covered queries are served from that cache. See +[Multiple Graphs → Caching](/memgraph-zero/memgql/multiple-graphs#caching-a-graph-in-memgraph). `TTL` is in seconds; `MAX_BYTES` accepts a plain byte count or a `K`/`M`/`G`/`T` binary suffix (e.g. `8G`). ``` -SHOW SCHEMA [FOR ]; -- unified routing index: labels, rel-types, properties -REFRESH SCHEMA; -- re-introspect live Cypher connections (Memgraph/Neo4j) +-- Introspection +SHOW CONNECTORS; -- registered connectors +SHOW CONNECTIONS; -- live connections +SHOW GRAPHS [ON CONNECTOR ]; -- registered graphs (name, connector(s), type, access, counts) +SHOW GRAPH ; -- one graph's details +SHOW MAPPINGS; -- per-graph mappings +SHOW SCHEMA [FOR ]; -- unified routing index: labels, rel-types, properties +EXPORT SCHEMA [TO '']; -- merged catalog as canonical schema JSON (round-trippable) +REFRESH SCHEMA; -- re-introspect live Cypher connections (Memgraph/Neo4j) ``` ``` @@ -119,7 +129,7 @@ USE UNION | UNION ALL | INTERSECT | INTERSECT ALL | EXCEPT | EXCEPT ALL USE ; --- USE-free: routes automatically when the query's labels / rel-types / +-- Routing: routes automatically when the query's labels / rel-types / -- properties match exactly one registered source (see Multiple Graphs). ; ``` @@ -129,7 +139,7 @@ schema signals (labels, relationship types, properties) match exactly one source; zero or multiple matches hard-error. Identifier matching is **exact** (`:person` ≠ `Person`). Memgraph sources must run with `--schema-info-enabled` for property-level introspection. See -[Multiple Graphs → USE-free routing](/memgraph-zero/memgql/multiple-graphs#use-free-routing). +[Multiple Graphs → Routing](/memgraph-zero/memgql/multiple-graphs#routing). ## Configuration Reference @@ -157,7 +167,7 @@ CLI flags on the Bolt server binary. #### Log Levels -Levels form a severity ladder — picking a level also emits everything more +Levels form a severity ladder; picking a level also emits everything more severe. Values are case-insensitive (`--log-level=debug` and `--log-level=DEBUG` are the same; `WARN` is accepted for `WARNING`). @@ -175,7 +185,7 @@ values. The `RUST_LOG` environment variable is not consulted. To see query traffic and what MemGQL translates it into, run with `--log-level=DEBUG`. The `info-queries-only` value from earlier releases was -removed in v0.7.0 — see the +removed in v0.7.0; see the [changelog](/memgraph-zero/memgql/changelog). ### Enterprise License @@ -233,40 +243,41 @@ connections. | Variable | Default | Description | |----------------|-----------------------------------------------------------------|---------------------------| | `POSTGRES_URL` | `host=localhost user=postgres password=postgres dbname=postgres` | libpq connection string | -| `MAPPING_FILE` | _(none, uses built-in default)_ | Path to JSON mapping file | +| `MAPPING_FILE` | _(required)_ | Path to JSON mapping file | #### MySQL (`mysql`) | Variable | Default | Description | |----------------|-----------------------------------------------|------------------------------------------| | `MYSQL_URL` | `mysql://root:mysql@localhost:3306/test` | MySQL connection URL (`mysql://user:pass@host:port/database`) | -| `MAPPING_FILE` | _(none, uses built-in default)_ | Path to JSON mapping file | +| `MAPPING_FILE` | _(required)_ | Path to JSON mapping file | #### Oracle (`oracle`) | Variable | Default | Description | |-----------------|----------------------------------------------------------|------------------------------------------| | `ORACLE_URL` | `oracle://system:oracle@localhost:1521/FREEPDB1` | Easy Connect URL (`oracle://user:pass@host:port/service_name`). The default targets [Oracle Database Free 23ai](https://www.oracle.com/database/free/) (service `FREEPDB1`). | -| `MAPPING_FILE` | _(none, uses built-in default)_ | Path to JSON mapping file (same format as Postgres) | +| `MAPPING_FILE` | _(required)_ | Path to JSON mapping file (same format as Postgres) | The Oracle connector uses [oracle-rs](https://crates.io/crates/oracle-rs), a pure-Rust implementation of Oracle's TNS wire protocol, pooled via [deadpool-oracle](https://crates.io/crates/deadpool-oracle). No OCI / ODPI-C -/ Instant Client is required at build or runtime — the bundled Docker image +/ Instant Client is required at build or runtime: the bundled Docker image ships only the bolt server binary plus TLS roots, and local builds work on macOS, Linux, and Windows without any extra system packages. #### SQL Server (`sqlserver`) -SQL Server has no standalone environment-variable mode — `CONNECTOR_TYPE=sqlserver` +SQL Server has no standalone environment-variable mode; `CONNECTOR_TYPE=sqlserver` is not supported. Register it at runtime in [`multi` mode](/memgraph-zero/memgql/multiple-graphs) with an ADO-style connection string: ```gql ADD CONNECTOR mssql TYPE sqlserver - URI 'Server=localhost,1433;Database=test;User Id=sa;Password=YourPassword;TrustServerCertificate=true' - MAPPING ; + URI 'Server=localhost,1433;Database=test;User Id=sa;Password=YourPassword;TrustServerCertificate=true'; +-- then map it into a graph +CREATE GRAPH sales FROM FILE '/data/sales.graph.json'; ``` `mssql`, `sql_server`, and `sql-server` are accepted aliases for the `sqlserver` @@ -277,7 +288,7 @@ type. See the [SQL Server connector page](/memgraph-zero/memgql/connect/sqlserve | Variable | Default | Description | |----------------|---------------------------------|---------------------------| | `DUCKDB_PATH` | `:memory:` | Path to DuckDB file | -| `MAPPING_FILE` | _(none, uses built-in default)_ | Path to JSON mapping file | +| `MAPPING_FILE` | _(required)_ | Path to JSON mapping file | #### ClickHouse (`clickhouse`) @@ -287,7 +298,7 @@ type. See the [SQL Server connector page](/memgraph-zero/memgql/connect/sqlserve | `CLICKHOUSE_USER` | `default` | ClickHouse user | | `CLICKHOUSE_PASS` | _(none)_ | ClickHouse password | | `CLICKHOUSE_DB` | `default` | ClickHouse database | -| `MAPPING_FILE` | _(none, uses built-in default)_ | Path to JSON mapping file | +| `MAPPING_FILE` | _(required)_ | Path to JSON mapping file | #### Apache Pinot (`pinot`) @@ -295,7 +306,7 @@ type. See the [SQL Server connector page](/memgraph-zero/memgql/connect/sqlserve |-----------------------|----------------------------|--------------------------------------------------| | `PINOT_URL` | `http://localhost:8099` | Pinot broker base URL or full SQL endpoint | | `PINOT_QUERY_OPTIONS` | `useMultistageEngine=true` | Query options sent with broker SQL requests | -| `MAPPING_FILE` | _(none, uses built-in default)_ | Path to JSON mapping file | +| `MAPPING_FILE` | _(required)_ | Path to JSON mapping file | #### Iceberg (`iceberg`) @@ -305,11 +316,11 @@ type. See the [SQL Server connector page](/memgraph-zero/memgql/connect/sqlserve | `TRINO_USER` | `trino` | Trino user | | `TRINO_CATALOG` | `iceberg` | Trino catalog | | `TRINO_SCHEMA` | `default` | Trino schema | -| `MAPPING_FILE` | _(none, uses built-in default)_ | Path to JSON mapping file | +| `MAPPING_FILE` | _(required)_ | Path to JSON mapping file | #### Iceberg Direct (`iceberg-direct`) -Native, in-process execution over Iceberg — reads the REST catalog and object +Native, in-process execution over Iceberg: reads the REST catalog and object storage (S3/MinIO) directly, no Trino. Read-only. | Variable | Default | Description | @@ -321,93 +332,26 @@ storage (S3/MinIO) directly, no Trino. Read-only. | `ICEBERG_DIRECT_S3_REGION` | `us-east-1` | S3 region | | `ICEBERG_DIRECT_S3_ACCESS_KEY_ID` | `admin` | S3/MinIO access key | | `ICEBERG_DIRECT_S3_SECRET_ACCESS_KEY` | `password` | S3/MinIO secret key | -| `MAPPING_FILE` | _(none, uses built-in default)_ | Path to JSON mapping file | +| `MAPPING_FILE` | _(required)_ | Path to JSON mapping file | S3 path-style access is always enabled (`s3.path-style-access=true`). ## Mapping Schema -The graph mapping file is a JSON document with two top-level arrays: -`nodes` and `edges`. MemGQL loads it either from the path in `MAPPING_FILE` -at startup or via the `ADD MAPPING FROM ''` statement at -runtime — both paths apply the same validation. A mapping with any -required field missing is rejected before the server begins serving -queries against it. - -```json -{ - "nodes": [ /* NodeMapping objects */ ], - "edges": [ /* EdgeMapping objects */ ] -} -``` +A graph's mapping declares its `vertices` and `edges`: labels and relationship +types mapped to backend tables (via `metaFields` and `attributes`) or to native +graph sources. The full format, with field tables and worked examples, lives on +the [Schema File](/memgraph-zero/memgql/schema-file) page. -### Node mapping fields - -| Field | Type | Required | Description | -|--------------|-------------------------|----------|-------------| -| `label` | string | yes | GQL node label (e.g. `"Person"`). Matched exactly (case-sensitive). | -| `table` | string | yes | Backend table or view name backing this label. | -| `id_column` | string | yes | Primary key column. Surfaces as GQL `id(n)` and is used to join across edges. | -| `properties` | object (string→string) | no | Map from GQL property name → backend column name. Properties not listed here pass through with the same name (`p.name` → column `name`). | - -Connector-specific fields: - -- **Iceberg** also accepts `catalog` and `schema_name` to fully qualify the - table reference (defaults: `"iceberg"`, `"default"`). -- **ClickHouse** also accepts `database` (default: `"default"`). - -### Edge mapping fields - -| Field | Type | Required | Description | -|------------------|-------------------------|----------|-------------| -| `rel_type` | string | yes | GQL relationship type (e.g. `"KNOWS"`). Matched exactly (case-sensitive). | -| `table` | string | yes | Junction or association table backing this edge type. | -| `id_column` | string | yes | Per-edge primary key. **Required since v0.6.** Carried through the recursive CTE's visited-edge set to enforce trail semantics on variable-length traversal. | -| `source_column` | string | yes | FK column pointing to the source node's `id_column`. | -| `target_column` | string | yes | FK column pointing to the target node's `id_column`. | -| `source_label` | string | yes | Label of the source node (must match a `label` declared in `nodes`). | -| `target_label` | string | yes | Label of the target node. | -| `properties` | object (string→string) | no | Map from GQL property name → column name on the edge table. | - -Connector-specific fields are the same as for nodes (`catalog` / `schema_name` -for Iceberg; `database` for ClickHouse). - -### Example - -```json -{ - "nodes": [ - { - "label": "Person", - "table": "persons", - "id_column": "id", - "properties": { "name": "full_name", "age": "age" } - }, - { - "label": "Company", - "table": "companies", - "id_column": "id" - } - ], - "edges": [ - { - "rel_type": "KNOWS", - "table": "knows", - "id_column": "id", - "source_column": "from_id", - "target_column": "to_id", - "source_label": "Person", - "target_label": "Person" - }, - { - "rel_type": "WORKS_AT", - "table": "works_at", - "id_column": "id", - "source_column": "person_id", - "target_column": "company_id", - "source_label": "Person", - "target_label": "Company" - } - ] -} -``` +The same mapping body is used everywhere a graph is defined: + +- **`--schema=`** at boot: connectors + graphs in one file. +- **`CREATE GRAPH FROM ''`** / **`FROM FILE ''`** at runtime. +- **`MAPPING_FILE`** in single-connector mode: a bare `{ "vertices": …, "edges": … }` + body (the connector comes from the environment). + +A mapping with a required field missing (for example a relational edge without +`metaFields.from` / `metaFields.to`) is rejected before the server serves +queries against it. The legacy `nodes` / `id_column` / `rel_type` format is no +longer accepted; loading it raises an actionable error pointing at the new +format. diff --git a/pages/memgraph-zero/memgql/schema-file.mdx b/pages/memgraph-zero/memgql/schema-file.mdx new file mode 100644 index 000000000..8f7239bca --- /dev/null +++ b/pages/memgraph-zero/memgql/schema-file.mdx @@ -0,0 +1,463 @@ +--- +title: Schema File +description: Boot MemGQL from a single JSON file that declares connectors and the named graphs mapped over them. +--- + +# Schema File + +MemGQL federates one or more backends behind a single Bolt endpoint and exposes +them as **named graphs** you query with GQL / openCypher. A schema file is the +one place that declares both halves of that setup: + +> **A connector is a connection. A graph is a mapping laid over connectors. +> You query graphs; routing figures out which backend to hit.** + +Everything in the file can also be created at runtime with +[DDL](/memgraph-zero/memgql/reference#graph-management-query-syntax); the schema +file is the boot-time equivalent, and runtime changes persist back to it. + +## Booting from a schema file + +Run MemGQL in Docker: mount the schema file and pass its container path. + +```bash +docker run --rm -p 7688:7688 \ + -e BOLT_LISTEN_ADDR=0.0.0.0:7688 \ + -v "$(pwd)/schema.json:/data/schema.json" \ + memgraph/memgql:latest --schema=/data/schema.json +``` + +`--schema` (or the `SCHEMA_FILE` env var) is all you need; it implies +multi-connector mode. Set `BOLT_LISTEN_ADDR=0.0.0.0:7688` so the Bolt port is +reachable from outside the container (the default binds loopback only), and run +MemGQL on the same Docker network as the backends its connectors reference. + +Every connector connects and every graph registers at boot, so no `CONNECT` or +`USE` is required to start querying. Runtime DDL writes the updated catalog back +to this file atomically, so a restart reloads the same state with no replay +(credentials stay on disk but are masked in `EXPORT SCHEMA`). + +The file is validated at boot. A malformed mapping (for example a relational +edge missing `metaFields.from` / `metaFields.to`, or the legacy `nodes` / +`id_column` format) is rejected with an actionable error before the server +starts serving. + +The file has two top-level sections: + +```json +{ + "connectors": [], + "graphs": [] +} +``` + +## A minimal schema + +The smallest useful file is one connector and one vertex: a label mapped to a +table by its primary-key column. + +```json +{ + "connectors": [ + { + "name": "pg", + "type": "postgres", + "connection": { + "uri": "postgresql://demo:demo@localhost:5432/demo" + } + } + ], + "graphs": [ + { + "name": "store", + "vertices": [ + { + "label": "Product", + "mappedTableSource": { + "connector": "pg", + "table": "products", + "metaFields": { + "id": "pid" + } + }, + "attributes": [ + { + "name": "pid", + "type": "Int" + }, + { + "name": "title" + } + ] + } + ] + } + ] +} +``` + +Boot it, then query the label by name: + +```gql +MATCH (p:Product) RETURN p.title LIMIT 5; +``` + +The sections below are the full field reference for each block. + +## `connectors` + +A connector is a **pure connection**: how to reach a backend, and nothing about +graph shape. `catalogs` is accepted as an alias for `connectors`. + +```json +{ + "connectors": [ + { + "name": "pg", + "type": "postgres", + "connection": { + "uri": "postgresql://user:pass@host:5432/db" + } + } + ] +} +``` + +| Field | Required | Description | +|--------------|----------|-------------| +| `name` | yes | Referenced by graphs and DDL. | +| `type` | yes | Backend type (see below). | +| `connection` | yes | A **native** connection block (see fields below). JDBC-style keys (`jdbc`, `driverClass`, …) are rejected. | + +Every `connection` field is optional; supply what a given backend needs: + +| `connection` field | Used for | +|--------------------|----------| +| `uri` | Connection string for the backend. | +| `user` / `password`| Credentials (override / complement the URI). | +| `database` | Database name. | +| `schema` | Middle namespace (PostgreSQL schema, MySQL / ClickHouse database, Iceberg schema). | +| `catalog` | Native root for three-level backends (Iceberg catalog, SQL Server database). | +| `path` | File-backed backends (DuckDB). | + +**Supported `type` values:** `memgraph`, `neo4j`, `postgres` (`postgresql`), +`mysql`, `sqlserver`, `oracle`, `duckdb`, `iceberg`, `iceberg-direct`, +`clickhouse`, `pinot`. + +## `graphs` + +Each graph is a set of `vertices` and `edges` laid over the shared connectors. A +single graph may span **several** connectors. + +```json +{ + "graphs": [ + { + "name": "store", + "vertices": [], + "edges": [] + } + ] +} +``` + +> **Single-graph shorthand:** put `vertices` / `edges` at the top level instead +> of a `graphs` block and you get one implicit graph named `default`. + +### Vertices + +A vertex maps a **label** to a backend source. Exactly one source kind is set: + +- **`mappedTableSource`**: a relational table (PostgreSQL, MySQL, Oracle, + SQL Server, DuckDB, ClickHouse, Iceberg). Rows become nodes. +- **`mappedGraphSource`**: a native graph backend (Memgraph / Neo4j). Queries + pass through as Cypher; only the connector is declared. + +```json +{ + "label": "Customer", + "mappedTableSource": { + "connector": "pg", + "schema": "public", + "table": "customers", + "metaFields": { + "id": "cid" + } + }, + "attributes": [ + { + "name": "cid", + "type": "Int" + }, + { + "name": "name", + "column": "full_name" + }, + { + "name": "email", + "type": "String" + } + ] +} +``` + +```json +{ + "label": "User", + "mappedGraphSource": { + "connector": "mg" + }, + "attributes": [ + { + "name": "uid", + "type": "Int" + }, + { + "name": "handle" + } + ] +} +``` + +| Field | Source kind | Description | +|-------|-------------|-------------| +| `mappedTableSource.connector` | relational | Connector name (alias `catalog`). | +| `mappedTableSource.schema` | relational | Optional middle namespace (PostgreSQL schema, etc.). | +| `mappedTableSource.table` | relational | The backing table. | +| `mappedTableSource.metaFields.id` | relational | Primary-key column; the node's identity and the join key for edges. | +| `mappedGraphSource.connector` | native | Connector name (Memgraph / Neo4j); the query passes through as Cypher. | + +### Edges + +An edge maps a **relationship type** between two labels. Relational edges name +the foreign-key columns via `metaFields.from` / `metaFields.to`; native-graph +edges pass through. + +```json +{ + "label": "PURCHASED", + "from": "Customer", + "to": "Product", + "mappedTableSource": { + "connector": "pg", + "table": "orders", + "metaFields": { + "id": "oid", + "from": "cid", + "to": "pid" + } + }, + "attributes": [ + { + "name": "qty", + "type": "Int" + } + ] +} +``` + +```json +{ + "label": "FOLLOWS", + "from": "User", + "to": "User", + "mappedGraphSource": { + "connector": "mg" + } +} +``` + +A relational edge adds two more `metaFields` on top of `id`: + +| `metaFields` key | Description | +|------------------|-------------| +| `id` | The edge row's own key. | +| `from` | Foreign-key column pointing at the **source** vertex's id column. | +| `to` | Foreign-key column pointing at the **target** vertex's id column. | + +`from` and `to` are **required** on relational edges; they define the traversal +`(from)-[:LABEL]->(to)`. Loading fails with an actionable error if either is +missing. Native-graph edges (`mappedGraphSource`) need only `from` / `to` +labels; the traversal is resolved by the backend. + +### Attributes + +`attributes` declare the properties a label exposes. + +| Field | Required | Description | +|----------|----------|-------------| +| `name` | yes | The GQL property name. | +| `column` | no | The backing column (defaults to `name`), e.g. property `name` ← column `full_name`. | +| `type` | no | Recorded, not enforced at query time (default `String`). | + +Allowed `type` values: `Boolean`, `Byte`, `Short`, `Int`, `Long`, `HugeInt`, +`Float`, `Double`, `Decimal`, `String`, `Date`, `DateTime`. + +`attributes` are optional and need not be exhaustive: a property you don't list +still resolves to a same-named column at query time (`p.sku` → column `sku`). +Declare the ones you want to **rename** (`column`), give a **type**, or route +queries by. + +> **Note:** for routing to match a query by property (e.g. `WHERE c.email = …`), +> the property must be a declared `attribute`. Route by label or relationship +> type, or add an explicit `USE`, when a property isn't declared. See +> [routing](/memgraph-zero/memgql/multiple-graphs#routing). + +## Complete example + +One engine over **Memgraph** (a social graph) and **PostgreSQL** (a store), +sharing a `uid` / `cid` id space so you can query each on its own or join across +them. + +```json +{ + "connectors": [ + { + "name": "mg", + "type": "memgraph", + "connection": { + "uri": "bolt://localhost:7687" + } + }, + { + "name": "pg", + "type": "postgres", + "connection": { + "uri": "postgresql://demo:demo@localhost:5432/demo" + } + } + ], + "graphs": [ + { + "name": "social", + "vertices": [ + { + "label": "User", + "mappedGraphSource": { + "connector": "mg" + }, + "attributes": [ + { + "name": "uid", + "type": "Int" + }, + { + "name": "handle" + }, + { + "name": "name" + } + ] + } + ], + "edges": [ + { + "label": "FOLLOWS", + "from": "User", + "to": "User", + "mappedGraphSource": { + "connector": "mg" + } + } + ] + }, + { + "name": "store", + "vertices": [ + { + "label": "Customer", + "mappedTableSource": { + "connector": "pg", + "table": "customers", + "metaFields": { + "id": "cid" + } + }, + "attributes": [ + { + "name": "cid", + "type": "Int" + }, + { + "name": "name", + "column": "full_name" + }, + { + "name": "email" + } + ] + }, + { + "label": "Product", + "mappedTableSource": { + "connector": "pg", + "table": "products", + "metaFields": { + "id": "pid" + } + }, + "attributes": [ + { + "name": "pid", + "type": "Int" + }, + { + "name": "title" + }, + { + "name": "price", + "type": "Double" + } + ] + } + ], + "edges": [ + { + "label": "PURCHASED", + "from": "Customer", + "to": "Product", + "mappedTableSource": { + "connector": "pg", + "table": "orders", + "metaFields": { + "id": "oid", + "from": "cid", + "to": "pid" + } + }, + "attributes": [ + { + "name": "qty", + "type": "Int" + } + ] + } + ] + } + ] +} +``` + +Boot it, then query by label: + +```gql +MATCH (u:User) RETURN u.handle, u.name ORDER BY u.uid LIMIT 5; +MATCH (c:Customer)-[:PURCHASED]->(p:Product) RETURN c.name, p.title; +MATCH (p:Product) WHERE p.price > 100 RETURN p.title ORDER BY p.price DESC; +``` + +The shared `uid` / `cid` id space also lets one query **join across both +backends**. See [Multiple Graphs](/memgraph-zero/memgql/multiple-graphs) for the +full query model (routing, cross-backend joins, and composites) and +[Reference](/memgraph-zero/memgql/reference#graph-management-query-syntax) for +the runtime DDL that builds the same catalog live. + +## How it works + +- **Connector = connection, graph = mapping.** A graph maps labels and + relationship types onto one or more connectors, so a single graph can span + backends. +- **Routing by label.** At boot (and on `REFRESH SCHEMA`) the engine tracks + which graph defines each label. A query with no `USE` routes by the labels and + declared properties it mentions; anything ambiguous returns an actionable + error rather than a silent guess. diff --git a/pages/memgraph-zero/memgql/use-cases/agentic.mdx b/pages/memgraph-zero/memgql/use-cases/agentic.mdx index 9b7cbde7b..101144bc3 100644 --- a/pages/memgraph-zero/memgql/use-cases/agentic.mdx +++ b/pages/memgraph-zero/memgql/use-cases/agentic.mdx @@ -42,19 +42,19 @@ behind a single GQL endpoint. Agents connect to one URL and query everything. Agents don't need hardcoded schemas or hand-written `USE` clauses. A single [`SHOW SCHEMA`](/memgraph-zero/memgql/multiple-graphs#schema-discovery) returns the unified index of every label, relationship type, and property across all -connected backends — the agent's map of what's queryable and where the +connected backends: the agent's map of what's queryable and where the entities connect. -From there, [USE-free routing](/memgraph-zero/memgql/multiple-graphs#use-free-routing) +From there, [schema-based routing](/memgraph-zero/memgql/multiple-graphs#routing) lets the agent write a plain GQL query and let the engine pick the backend from the labels and properties it referenced: ```gql -- The agent doesn't need to know `name` lives in PostgreSQL or that --- FRIEND_OF lives in Memgraph — both route automatically. +-- FRIEND_OF lives in Memgraph; both route automatically. MATCH (p:Person {id: 1})-[:FRIEND_OF]->(f:Person) RETURN f.name; ``` Routing is strict: if a query is ambiguous or matches nothing, MemGQL returns an actionable error that names the candidate sources and points back at -`SHOW SCHEMA` — so an agent can self-correct instead of failing silently. +`SHOW SCHEMA`, so an agent can self-correct instead of failing silently. diff --git a/pages/memgraph-zero/memgql/use-cases/public-private.mdx b/pages/memgraph-zero/memgql/use-cases/public-private.mdx index 87be6d1ec..f4aa979ce 100644 --- a/pages/memgraph-zero/memgql/use-cases/public-private.mdx +++ b/pages/memgraph-zero/memgql/use-cases/public-private.mdx @@ -88,10 +88,7 @@ services: - -c - | echo "ADD CONNECTOR mg TYPE memgraph URI 'memgraph:7687' GRAPH memgraph;" | mgconsole --host memgql --port 7688 && - echo "CONNECT mg AS mg_conn;" | mgconsole --host memgql --port 7688 && - echo "ADD MAPPING pg_social FROM '/mappings/pg_mapping.json';" | mgconsole --host memgql --port 7688 && - echo "ADD CONNECTOR pg TYPE postgres URI 'host=postgres user=postgres password=postgres dbname=postgres' MAPPING pg_social;" | mgconsole --host memgql --port 7688 && - echo "CONNECT pg AS pg_conn;" | mgconsole --host memgql --port 7688 && + echo "ADD CONNECTOR pg TYPE postgres URI 'host=postgres user=postgres password=postgres dbname=postgres';" | mgconsole --host memgql --port 7688 && echo "MemGQL bootstrap complete." depends_on: memgql: @@ -145,9 +142,10 @@ CREATE TABLE users ( ); CREATE TABLE friend_of ( + id SERIAL PRIMARY KEY, user_id INTEGER NOT NULL REFERENCES users(id), friend_id INTEGER NOT NULL REFERENCES users(id), - PRIMARY KEY (user_id, friend_id) + UNIQUE (user_id, friend_id) ); EOF ``` @@ -159,26 +157,40 @@ Tell MemGQL how the relational tables map to graph labels: ```bash cat > pg_mapping.json << 'EOF' { - "nodes": [ + "vertices": [ { "label": "User", - "table": "users", - "id_column": "id", - "properties": { - "name": "name", - "email": "email" - } + "mappedTableSource": { + "connector": "pg", + "table": "users", + "metaFields": { + "id": "id" + } + }, + "attributes": [ + { + "name": "name" + }, + { + "name": "email" + } + ] } ], "edges": [ { - "rel_type": "FRIEND_OF", - "table": "friend_of", - "id_column": "id", - "source_column": "user_id", - "target_column": "friend_id", - "source_label": "User", - "target_label": "User" + "label": "FRIEND_OF", + "from": "User", + "to": "User", + "mappedTableSource": { + "connector": "pg", + "table": "friend_of", + "metaFields": { + "id": "id", + "from": "user_id", + "to": "friend_id" + } + } } ] } @@ -213,8 +225,16 @@ SHOW CONNECTIONS; Register graphs in the catalog so they can be referenced by name in composite queries: ```gql -ADD GRAPH public ON CONNECTOR mg GRAPH memgraph; -ADD GRAPH private ON CONNECTOR pg MAPPING pg_social; +CREATE GRAPH public FROM '{ + "vertices": [ + { "label": "Product", "mappedGraphSource": { "connector": "mg" } }, + { "label": "Category", "mappedGraphSource": { "connector": "mg" } } + ], + "edges": [ + { "label": "IN_CATEGORY", "from": "Product", "to": "Category", "mappedGraphSource": { "connector": "mg" } } + ] +}'; +CREATE GRAPH private FROM FILE '/mappings/pg_mapping.json'; ``` Verify the graphs: @@ -225,11 +245,7 @@ SHOW GRAPHS; ## 7. Seed public data in Memgraph -Insert a public product catalog: - -```gql -SET DEFAULT CONNECTION mg_conn; -``` +Insert a public product catalog; the `Product` / `Category` labels route to `public`: ```gql INSERT @@ -255,11 +271,7 @@ MATCH (p:`Product`)-[:IN_CATEGORY]->(c:Category) RETURN p.name, c.name; ## 8. Seed private data in PostgreSQL -Switch to the private backend and insert customer profiles: - -```gql -SET DEFAULT CONNECTION pg_conn; -``` +Insert customer profiles; the `User` label routes to `private`: ```gql INSERT (alice:User {name: "Alice", email: "alice@acme.com"}); @@ -294,9 +306,9 @@ single composite query that combines results from the public product catalog and the private customer directory: ```gql -USE public MATCH (p:`Product`) RETURN p.name AS name +MATCH (p:`Product`) RETURN p.name AS name UNION ALL -USE private MATCH (u:User) RETURN u.name AS name; +MATCH (u:User) RETURN u.name AS name; ``` Each branch is routed to its respective backend and the results are combined @@ -314,23 +326,18 @@ locally. Standard GQL composite operators (`UNION`, `UNION ALL`, `INTERSECT`, ## Cleanup -Remove the graph catalog entries: +Clear the seeded data (through the graphs, before dropping them): ```gql -UNADD GRAPH public; -UNADD GRAPH private; +USE public MATCH (n) DETACH DELETE n; +USE private MATCH (n) DETACH DELETE n; ``` -Clear the public data: - -```gql -USE CONNECTION mg_conn MATCH (n) DETACH DELETE n; -``` - -Clear the private data: +Remove the graph catalog entries: ```gql -USE CONNECTION pg_conn MATCH (n) DETACH DELETE n; +DROP GRAPH public; +DROP GRAPH private; ``` Stop and remove the stack: