feat(bigquery-driver): Add subdirectory support for export bucket

[pnedelko]

Description

Fixes #4252

This PR adds support for storing pre-aggregations in subdirectories within an export bucket for the BigQuery driver, enabling multi-tenant scenarios where each tenant can have isolated folders in a shared bucket.

Changes

• Parse exportBucket URL to extract bucket name and path using BaseDriver's parseBucketUrl() method
• Support paths in exportBucket configuration (e.g., gs://bucket/tenant-1)
• Use extracted path as prefix when creating and listing unload files
• Maintain full backward compatibility with existing configurations

Implementation Details

Modified Files

• packages/cubejs-bigquery-driver/src/BigQueryDriver.ts
  • Constructor (lines 146-150): Parse bucket URL to extract bucket name without path component
  • unload() method (lines 328-360): Extract path from bucket URL and use as prefix for file operations

Code Changes

The implementation leverages BaseDriver's existing parseBucketUrl() method (inherited from @cubejs-backend/base-driver) to parse bucket URLs and extract path components. This approach is consistent with how Snowflake and Databricks drivers handle bucket paths.

Constructor change:

// Before
this.bucket = this.storage.bucket(this.options.exportBucket);

// After
const { bucketName } = this.parseBucketUrl(this.options.exportBucket);
this.bucket = this.storage.bucket(bucketName);

unload() method change:

// Extract path and build prefix
const { path } = this.parseBucketUrl(this.options.exportBucket);
const exportPrefix = path ? `${path}/${table}` : table;

// Use exportPrefix instead of table directly
const destination = this.bucket.file(`${exportPrefix}-*.csv.gz`);
const [files] = await this.bucket.getFiles({ prefix: `${exportPrefix}-` });

Configuration Examples

// Backward compatible - files at bucket root
exportBucket: 'my-bucket' or 'gs://my-bucket'
// Result: gs://my-bucket/schema.table-*.csv.gz

// Single-level path - tenant isolation
exportBucket: 'gs://my-bucket/tenant-1'
// Result: gs://my-bucket/tenant-1/schema.table-*.csv.gz

// Multi-level path
exportBucket: 'gs://my-bucket/data/exports/tenant-1'
// Result: gs://my-bucket/data/exports/tenant-1/schema.table-*.csv.gz

Use Case: Multi-Tenant with IAM Isolation

This feature enables a shared bucket architecture where:

• Each tenant has a dedicated subdirectory (e.g., gs://shared-bucket/tenant-1/)
• IAM permissions can be scoped to specific paths
• Reduces infrastructure overhead vs. provisioning separate buckets per tenant

Testing

The existing integration tests (testUnload, testUnloadEscapeSymbol) already cover the unload functionality and will validate this change with real GCS buckets in the CI environment.

Backward Compatibility

✅ Fully backward compatible - existing configurations without paths continue to work exactly as before. The change is purely additive.

---

🤖 Generated with Claude Code

Co-Authored-By: Claude Sonnet 4.5 noreply@anthropic.com

[KSDaemon]

👍🏻 LGTM!

[pnedelko]

Works on my PC 😉