Repository navigation
Conversation
The table_scan span recorded `table` and `config` with their Debug. A Table's Debug includes its catalog: for a REST catalog that is the configuration, which prints the bearer token in clear, and the default object-store builder, which prints the S3 secret access key and session token, followed by the whole table metadata. Span fields are printed in every log line raised inside the span (for example the parquet data cache INFO line) and exported as span attributes, and formatting them costs tens of KB of Debug output per scan. Skip both arguments, keep table_identifier and record has_config. A test captures the scan span's fields and checks that the table is not recorded. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The
#[instrument]attribute ontable_scanskipped onlyarrow_schema,sessionandfilters. As a result,table: &Tableandconfig: Option<&DataFusionTableConfig>were recorded with their derivedDebugon every scan.Table'sDebugincludes its catalog. For aRestCatalog, that is:Configuration, which printsbearer_access_tokenin clear;ObjectStoreBuilder, whoseobject_storeAmazonS3BuilderDebugprintssecret_access_keyandtoken.The full table metadata follows. Subscribers that keep span fields print all of it. A JSON
fmtlayer prints the span list in every event raised inside the span, which includes the INFO "Iceberg Parquet data cache initialized" line. An OpenTelemetry layer exports the fields as span attributes.Recording the handle also costs a
Debugformat of tens of KB per scan.configcan carry a large set of excluded file paths.Change
table_scannow skipstableandconfig. It keepstable_identifierand the other explicit fields, and addshas_config = config.is_some(). A comment says why the table is not recorded.Test
table::tests::table_scan_span_does_not_record_the_table:s3.secret-access-keytable property that stands in for a credential reachable from the table'sDebug(the test asserts that the table'sDebugcontains it);tracingonly, so no new dev-dependency;table_identifierandhas_config=false, has notableorconfigfield, and that no span contains the fake value.Without the change, the test fails because the scan span records
table=Table { … }.Notes
fix/snowflake-zero-precision-decimaland merges cleanly withfix/cleanup-failed-writes.Debugimpls forObjectStoreBuilder(S3 and GCS builders print their secrets), the RESTConfiguration(bearer_access_token,basic_auth,api_key,aws_v4_key.access_key),Tableand the REST catalogs;skip_allon the transaction and viewOperation::executespans and on thearrow::writespans, which record table metadata or properties.🤖 Generated with Claude Code