SQL GA - Get started#571
Conversation
✅ Deploy Preview for rp-cloud ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
andrewhsu
left a comment
andrewhsu
left a comment
There was a problem hiding this comment.
CREATE TABLE command needs to change to match web ui
kbatuigas
force-pushed
the
DOC-1856-document-feature-sql-engine-deployment-in-cloud
branch
from
ae3ae40 to
2dd0c5a
Compare
kbatuigas
force-pushed
the
DOC-1856-document-feature-sql-engine-deployment-in-cloud
branch
from
2dd0c5a to
fb15751
Compare
|
|
||
| To enable, scale, or disable SQL on an existing cluster, you also need the cluster ID, which you can find in the *Details* section of the cluster overview in the Cloud Console. | ||
|
|
||
| // TODO: Confirm UI functionality |
There was a problem hiding this comment.
@c-julin should have knowledge about this matter
| -H "Content-Type: application/json" | ||
| ---- | ||
| + | ||
| When the operation is complete, the response shows `"state": "STATE_COMPLETED"`. |
There was a problem hiding this comment.
We need to talk a bit more about it, I'll send you a link to slack thread.
There was a problem hiding this comment.
Added some guidance here https://github.com/redpanda-data/cloud-docs/pull/571/changes#diff-53e047807d28a0255b66ea613665a131405259a0926d855ba2b3ab3dd13fbe5bR129 but this is based on current state afaik, we can update later if the UX is changed
There was a problem hiding this comment.
you should sync with @c-julin on this one :)
|
|
||
| Redpanda SQL supports horizontal scaling from 1 to 9 nodes per cluster. Scaling to 0 is not supported. To remove Redpanda SQL from a cluster, disable the SQL engine instead. | ||
|
|
||
| // TODO: Confirm UI functionality |
|
|
||
| == Verify the SQL engine is running | ||
|
|
||
| After you enable Redpanda SQL, the cluster overview page in the Cloud console shows the *SQL* tab. The *Details* pane also displays the number of SQL nodes deployed with the cluster. Provisioning may take several minutes. |
There was a problem hiding this comment.
30mins is a safe boundary here imho
|
|
||
| [,sql] | ||
| ---- | ||
| SHOW NODES; -- List SQL compute nodes and their status |
There was a problem hiding this comment.
I'll think adding some queries related to CATALOG objectst may be a good idea, someone from Oxla core should be able to provide insights on this matter.
| * https://www.postgresql.org/download/[`psql`^] (PostgreSQL client) installed on your local machine. | ||
|
|
||
| // TODO: Verify the exact connection string format and where users get credentials. | ||
| // From PRD: SCRAM auth preserved, connection string available in Cloud Console and API response. |
There was a problem hiding this comment.
This is 'the legacy auth method' that Oxla is preserving for tools like PowerBI etc. @grzebiel / @paulzhang97 can provide more insights.
There was a problem hiding this comment.
We've discussed it f2f. Should be clear now.
TLDR: connection string is availble in console. rpk version supporting rpk claud auth token required
There was a problem hiding this comment.
Connection string available in Console, mentioned in this step https://github.com/redpanda-data/cloud-docs/pull/571/changes#diff-0aaa7195ceaa99e1ae1dfb70388ac642bf4af38bba78aa1f6d43c1d1a9342e44R115
legacy connections discussed in further detail https://github.com/redpanda-data/cloud-docs/pull/580/changes#diff-464751f0d8b4943851c484fba03b6b5a9325c429f993852df10f91f31e7eb24bR119 please also have a look @grzebiel
|
|
||
| Redpanda SQL is deny-all by default. All queries to Redpanda SQL run under a single super-user SASL credential associated with `default_redpanda_catalog`, but the engine enforces per-user table access through Redpanda SQL `GRANT` statements. To let a non-admin user query a table, the admin creates a role and grants access: | ||
|
|
||
| // TODO: Confirm the exact GRANT mechanism with engineering. Open questions: |
There was a problem hiding this comment.
discussed on f2f meeting.
TLDR: two permissions available in the RBAC: dataplane->sql->{access, manage}.
manage -> sql superuser able to read and grand accessess to all the topics/tables
access -> regular user. No access to any catalog unless granted by superuser.
Standard sql grant system to manage access to catalogs
There was a problem hiding this comment.
Access mechanism explained in separate PR https://github.com/redpanda-data/cloud-docs/pull/580/changes
Updates the Enable Redpanda SQL page so the horizontal-scaling range is 1 to 9 nodes (down from 1 to 12) per the PM SME 2026-05-07. Adds a TODO on the Disable Redpanda SQL section listing the specific behaviors to confirm with engineering before publication: what state is purged, what is deleted from object storage, the effect on topic and Schema Registry data, in-flight query error behavior, and re-enable semantics. Updates the quickstart so it explicitly targets an admin who can view the SQL connection details in the Cloud Console, and adds an optional admin-run mini-section showing how to grant a non-admin user SELECT access on a Redpanda SQL table. Includes a TODO referencing qa-questions.md #21 to refine the role/grant example once engineering confirms whether the SQL GRANT statement alone enforces per-user table access or whether an additional ACL/Kafka-side step is required. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Andrew Hsu <xuzuan@gmail.com>
kbatuigas
force-pushed
the
DOC-1856-document-feature-sql-engine-deployment-in-cloud
branch
from
2731f43 to
a0b51f4
Compare
|
Hey Kat, just FYI we now have a settings page UI for enabling/scaling/disabling oxla on an existing cluster so we could upadrte this page with screenshots instead of getting the user to run curl api requests. The UI is still changing but it is there |
|
It would also be useful to call out RPSQL is only available for UBB based orgs |
| Skip this section if you already have a Redpanda topic with a schema registered in Schema Registry that you want to query. | ||
| ==== | ||
|
|
||
| If you don't have a schema-registered topic to query yet, follow these steps to create an `orders` topic with a small set of sample records. Redpanda SQL reads the topic's schema from Schema Registry to map fields to SQL columns, so the topic must have a registered schema before you can query it. |
There was a problem hiding this comment.
@kbatuigas this wording doesn't compute for me. You mean : if you don't have a topic with a corresponding schema in the schema registry ....
There was a problem hiding this comment.
Rephrased to
If you don't have a topic with a corresponding schema in Schema Registry yet, follow these steps to create an
orderstopic with a small set of sample records.
| + | ||
| Redpanda SQL reads the registered schema and maps each top-level field to a SQL column. | ||
| + | ||
| Records produced through the Cloud Console don't carry the Schema Registry wire-format prefix, so the example sets `confluent_wire_protocol = 'false'`. If your producer client adds the wire format, set this option to `'true'` or omit it. For details on this and other CREATE TABLE options, see xref:reference:sql/sql-statements/create-table.adoc[CREATE TABLE]. |
There was a problem hiding this comment.
Don't we have some tech debt here @weeco @r-vasquez where Console doesn't/can't use wire format but RPK does by default. Seems like it would be worth fixing so we can do both? That said there is a new header-based option for storing schema IDs in Confluent FYI
jacekseligarp
left a comment
jacekseligarp
left a comment
There was a problem hiding this comment.
Cloud part looks fine.
Let's not forget about tf-provider:
https://docs.redpanda.com/redpanda-cloud/manage/terraform-provider/
Once it lands ofc.
6 participants
Description
This pull request adds comprehensive "Get Started" documentation for Redpanda SQL, including new guides for enabling, connecting to, and using Redpanda SQL on BYOC clusters. The changes introduce a new navigation structure and detailed step-by-step instructions for both the Cloud Console and API workflows, as well as a hands-on quickstart for querying streaming data.
New Redpanda SQL onboarding documentation:
index.adoc), a quickstart guide for connecting and querying withpsql(sql-quickstart.adoc), and a detailed guide for enabling, scaling, and disabling Redpanda SQL on BYOC clusters (deploy-sql-cluster.adoc). [1] [2] [3]Navigation and structure updates:
nav.adocto include links to the new "Get Started," "Quickstart," and "Enable Redpanda SQL" guides under the Redpanda SQL section, improving discoverability and organization of onboarding content.Resolves https://github.com/redpanda-data/documentation-private/issues/
Review deadline: 19 May
Page previews
https://deploy-preview-571--rp-cloud.netlify.app/redpanda-cloud/sql/get-started/sql-quickstart/
https://deploy-preview-571--rp-cloud.netlify.app/redpanda-cloud/sql/get-started/deploy-sql-cluster/
Checks