-
Notifications
You must be signed in to change notification settings - Fork 6.7k
docs: add bigframes_queries sample snippets #14429
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
cf7d2cd
154eed9
b4f1c86
d2d3d25
7348654
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
| @@ -0,0 +1,139 @@ | ||||||
| # Copyright 2026 Google LLC | ||||||
| # | ||||||
| # Licensed under the Apache License, Version 2.0 (the "License"); | ||||||
| # you may not use this file except in compliance with the License. | ||||||
| # You may obtain a copy of the License at | ||||||
| # | ||||||
| # http://www.apache.org/licenses/LICENSE-2.0 | ||||||
| # | ||||||
| # Unless required by applicable law or agreed to in writing, software | ||||||
| # distributed under the License is distributed on an "AS IS" BASIS, | ||||||
| # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||||||
| # See the License for the specific language governing permissions and | ||||||
| # limitations under the License. | ||||||
|
|
||||||
|
|
||||||
| def query_standard_sql(): | ||||||
| # [START bigquery_bigframes_query] | ||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The Cloud samples rubric has the region tags outside of the function and move the imports to outside of the function. Refer to https://github.com/GoogleCloudPlatform/python-docs-samples/blob/main/AUTHORING_GUIDE.md |
||||||
| import bigframes.pandas as bpd | ||||||
|
|
||||||
| # Set partial ordering mode as the default configuration for BigQuery DataFrames. | ||||||
| bpd.options.bigquery.ordering_mode = "partial" | ||||||
|
|
||||||
| sql = """ | ||||||
| SELECT name FROM `bigquery-public-data.usa_names.usa_1910_current` | ||||||
| WHERE state = 'TX' | ||||||
| LIMIT 100 | ||||||
| """ | ||||||
|
|
||||||
| # Run a query alongside existing SQL. The project will be determined from default credentials. | ||||||
| df = bpd.read_gbq(sql) | ||||||
|
|
||||||
| # Run a query after explicitly specifying a project. | ||||||
| project_id = "your-project-id" | ||||||
| bpd.options.bigquery.project = project_id | ||||||
| df = bpd.read_gbq(sql) | ||||||
| # [END bigquery_bigframes_query] | ||||||
| return df | ||||||
|
|
||||||
|
|
||||||
| def query_legacy_sql(): | ||||||
| # [START bigquery_bigframes_query_legacy] | ||||||
| import bigframes.pandas as bpd | ||||||
|
|
||||||
| # Set partial ordering mode as the default configuration for BigQuery DataFrames. | ||||||
| bpd.options.bigquery.ordering_mode = "partial" | ||||||
|
|
||||||
| sql = """ | ||||||
| SELECT name FROM [bigquery-public-data:usa_names.usa_1910_current] | ||||||
| WHERE state = 'TX' | ||||||
| LIMIT 100 | ||||||
| """ | ||||||
|
|
||||||
| # Run a query using legacy SQL syntax. | ||||||
| query_config = {"query": {"useLegacySql": True}} | ||||||
| df = bpd.read_gbq(sql, configuration=query_config) | ||||||
| # [END bigquery_bigframes_query_legacy] | ||||||
| return df | ||||||
|
Comment on lines
+40
to
+57
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I update the error message to avoid confusion |
||||||
|
|
||||||
|
|
||||||
| def query_bqstorage(): | ||||||
| # [START bigquery_bigframes_query_bqstorage] | ||||||
| import bigframes.pandas as bpd | ||||||
|
|
||||||
| # Set partial ordering mode as the default configuration for BigQuery DataFrames. | ||||||
| bpd.options.bigquery.ordering_mode = "partial" | ||||||
|
|
||||||
| sql = """ | ||||||
| SELECT name FROM `bigquery-public-data.usa_names.usa_1910_current` | ||||||
| WHERE state = 'TX' | ||||||
| LIMIT 100 | ||||||
| """ | ||||||
|
|
||||||
| # Read query results into a server-side DataFrame without downloading data. | ||||||
| df = bpd.read_gbq(sql) | ||||||
|
|
||||||
| # When downloading results to an in-memory pandas DataFrame, bigquery-dataframes | ||||||
| # automatically uses the BigQuery Storage API if installed. | ||||||
| pandas_df = df.to_pandas() | ||||||
| # [END bigquery_bigframes_query_bqstorage] | ||||||
| return pandas_df | ||||||
|
|
||||||
|
|
||||||
| def query_parameters(): | ||||||
| # [START bigquery_bigframes_query_parameters] | ||||||
| import bigframes.pandas as bpd | ||||||
|
|
||||||
| # Set partial ordering mode as the default configuration for BigQuery DataFrames. | ||||||
| bpd.options.bigquery.ordering_mode = "partial" | ||||||
|
|
||||||
| sql = """ | ||||||
| SELECT name FROM `bigquery-public-data.usa_names.usa_1910_current` | ||||||
| WHERE state = @state | ||||||
| LIMIT 100 | ||||||
| """ | ||||||
|
|
||||||
| query_config = { | ||||||
| "query": { | ||||||
| "parameterMode": "NAMED", | ||||||
| "queryParameters": [ | ||||||
| { | ||||||
| "name": "state", | ||||||
| "parameterType": {"type": "STRING"}, | ||||||
| "parameterValue": {"value": "TX"}, | ||||||
| } | ||||||
| ], | ||||||
| } | ||||||
| } | ||||||
|
|
||||||
| df = bpd.read_gbq(sql, configuration=query_config) | ||||||
| # [END bigquery_bigframes_query_parameters] | ||||||
| return df | ||||||
|
|
||||||
|
|
||||||
| def upload_from_dataframe(): | ||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Parameterize the
Suggested change
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This test case is for adding samples in welcome page. No past in variables should be allowed.
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Gemini is correct. |
||||||
| # [START bigquery_bigframes_upload_from_dataframe] | ||||||
| import pandas as pd | ||||||
|
|
||||||
| import bigframes.pandas as bpd | ||||||
|
|
||||||
| # Set partial ordering mode as the default configuration for BigQuery DataFrames. | ||||||
| bpd.options.bigquery.ordering_mode = "partial" | ||||||
|
|
||||||
| # Create a local pandas DataFrame. | ||||||
| df = pd.DataFrame( | ||||||
| { | ||||||
| "my_string": ["a", "b", "c"], | ||||||
| "my_int64": [1, 2, 3], | ||||||
| "my_float64": [4.0, 5.0, 6.0], | ||||||
| } | ||||||
| ) | ||||||
|
|
||||||
| # Convert the local pandas DataFrame to a BigQuery DataFrame. | ||||||
| bq_df = bpd.read_pandas(df) | ||||||
|
|
||||||
| # Write the DataFrame to a BigQuery table. | ||||||
| table_id = "your-project.your_dataset.your_table_name" | ||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I prefer the original line better |
||||||
| bq_df.to_gbq(table_id, if_exists="replace") | ||||||
| # [END bigquery_bigframes_upload_from_dataframe] | ||||||
| return bq_df | ||||||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,50 @@ | ||
| # Copyright 2026 Google LLC | ||
| # | ||
| # Licensed under the Apache License, Version 2.0 (the "License"); | ||
| # you may not use this file except in compliance with the License. | ||
| # You may obtain a copy of the License at | ||
| # | ||
| # http://www.apache.org/licenses/LICENSE-2.0 | ||
| # | ||
| # Unless required by applicable law or agreed to in writing, software | ||
| # distributed under the License is distributed on an "AS IS" BASIS, | ||
| # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| # See the License for the specific language governing permissions and | ||
| # limitations under the License. | ||
|
|
||
| import bigframes_queries | ||
| import pytest | ||
|
|
||
|
|
||
| @pytest.mark.skip( | ||
| reason="Placeholder project ID 'your-project-id' cannot be executed by pytest, but snippet is required for welcome page documentation." | ||
| ) | ||
|
Comment on lines
+19
to
+21
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Don't skip this. |
||
| def test_query_standard_sql(): | ||
| df = bigframes_queries.query_standard_sql() | ||
| assert df is not None | ||
|
|
||
|
|
||
| @pytest.mark.skip( | ||
| reason="Legacy SQL syntax is not supported for execution by BigQuery DataFrames, but snippet is required for welcome page documentation." | ||
| ) | ||
| def test_query_legacy_sql(): | ||
| df = bigframes_queries.query_legacy_sql() | ||
| assert df is not None | ||
|
Comment on lines
+27
to
+32
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Agreed with Gemini. It's better not to have a sample than to have one that doesn't work. |
||
|
|
||
|
|
||
| def test_query_bqstorage(): | ||
| pandas_df = bigframes_queries.query_bqstorage() | ||
| assert pandas_df is not None | ||
|
|
||
|
|
||
| def test_query_parameters(): | ||
| df = bigframes_queries.query_parameters() | ||
| assert df is not None | ||
|
|
||
|
|
||
| @pytest.mark.skip( | ||
| reason="Requires a writable destination table so pytest skips execution, but snippet is required for welcome page documentation." | ||
| ) | ||
|
Comment on lines
+45
to
+47
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Don't skip this. Instead, parametrize the sample and pass in a writable destination. |
||
| def test_upload_from_dataframe(): | ||
| bq_df = bigframes_queries.upload_from_dataframe() | ||
| assert bq_df is not None | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
To make this snippet testable and avoid global side effects, parameterize the
project_idin the function signature with a default value of"your-project-id". This allows tests to pass a valid project ID while keeping the placeholder for documentation.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This test case is for adding samples in welcome page. No past in variables should be allowed.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Gemini is correct in this case.