feat(storage): implement V4 Signed Policy Documents by xlai20 · Pull Request #5914 · googleapis/google-cloud-rust

xlai20 · 2026-06-18T03:29:24Z

Description

This PR introduces support for generating V4 Signed Policy Documents (PostPolicyV4Builder), bringing the Rust SDK into feature parity with other official GCS SDKs for POST form object uploads.

Design Doc: go/rust-sdk-feature-v4-signed-policy-document-implementation_plan

Key Additions

PostPolicyV4Builder API: A fluent builder API to configure URL styles, exact-match fields, starts-with prefix conditions, and content-length-range limits.
Strict JSON Serialization: Implemented exact-byte character escaping using UTF-16 surrogate pairs (\uXXXX) to guarantee flawless Base64 JSON encoding, satisfying GCS's stringent signature validation requirements.

Testing & Validation

Manual Validation: I have manually run the added generate_signed_post_policy_v4 sample code against a real Cloud Storage bucket and verified the upload worked correctly.
Conformance Tests Passed: Integrated with the official v4_signatures.json test harness, successfully passing 100% (11/11) of the postPolicyV4Tests (covering Path Style, Virtual Hosted Style, Bucket Bound Hostnames, and Unicode conditions).
Cross-Language Parity Check: Verified that $key condition formulation and the maximum 7-day expiration bounds behave identically to the official Go and C++ SDKs.
Integration Tests: Added signed_post_policies_v4 in tests/storage/src/lib.rs which dynamically constructs a policy and performs a live upload using a reqwest multipart form. It ran successfully against a real Cloud bucket.

This reverts commit eded1b1.

gemini-code-assist

Code Review

This pull request implements support for GCS V4 Signed Policy Documents (POST Object Forms) by introducing PostPolicyV4Builder and PostPolicyV4Result, along with associated examples, integration tests, and conformance tests. Key feedback includes addressing a bug where custom endpoints with ports lose their port number during URL resolution, automatically prepending $ to starts-with fields, validating expiration limits and content length ranges, avoiding an unnecessary clone of client_email, and switching from HashMap to BTreeMap for deterministic field ordering.

codecov · 2026-06-18T03:38:20Z

Codecov Report

❌ Patch coverage is 91.19804% with 36 lines in your changes missing coverage. Please review.
✅ Project coverage is 97.85%. Comparing base (03f5ffa) to head (50c0664).
⚠️ Report is 5 commits behind head on main.

Files with missing lines	Patch %	Lines
src/storage/src/storage/post_policy.rs	91.19%	36 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #5914      +/-   ##
==========================================
- Coverage   97.90%   97.85%   -0.05%     
==========================================
  Files         234      235       +1     
  Lines       59422    59831     +409     
==========================================
+ Hits        58176    58548     +372     
- Misses       1246     1283      +37

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

joshuatants · 2026-06-18T11:02:41Z

+        let mut fields = BTreeMap::new();
+        fields.insert("key".to_string(), self.object.clone());
+        fields.insert(
+            "x-goog-algorithm".to_string(),
+            "GOOG4-RSA-SHA256".to_string(),
+        );
+        fields.insert("x-goog-credential".to_string(), credential);
+        fields.insert("x-goog-date".to_string(), request_timestamp);
+        fields.insert("x-goog-signature".to_string(), signature_hex);
+        fields.insert("policy".to_string(), encoded_policy);
+
+        // Add user-supplied fields (including custom metadata or x-ignore- fields)
+        for (key, value) in &self.fields {
+            fields.insert(key.clone(), value.clone());
+        }


What happens if we have

PostPolicyV4Builder::for_object(bucket, "object") .with_field("key", "another_object");

Looking at the code right now, I think that what would happen is that the signing happens with the original "object" (since user supplied fields with reserved keywords are ignored when building conditions), but lines 310-312 end up mapping key to another_object, meaning that the signature no longer matches the contents of the form. GCS will probably reject this?

Compare with SignedUrlBuilder (

google-cloud-rust/src/storage/src/storage/signed_url.rs

Line 534 in 03f5ffa

let mut query_parameters = self.query_parameters;

): there, the user supplied fields are added first, then the reserved parameters. This has the effect that the user cannot override the reserved fields, even if they try to do so.

Perhaps the safest way is to, in both SignedUrlBuilder and here, maintain a list of reserved keywords and reject with an error if the user tries to use one. We can leave that decision till later since we'll have to update SignedUrlBuilder.

Done.

I had made some changes to this sign_with method to address your concerns as follows:

In the output fields, I change the order in which reserved fields and user supplied fields are being added. Hence, now, the post_policy sign_with method is handling the output fields like what signed_url is handling the query paramters, that user cannot override the reserved fields. This also ensures that the signature will match the contents of the output form.

Yes, I am maintaining a list of reserved keywords and does not add to the input JSON document at all (there is a strict order requirement in the JSON input document before signing, so not adding them in the beginning is the best solution). Based on my research with other SDK implementation and a cross-check with the Rust SDK community, a silent ignoring here is good enough; we don't need to explicitly reject user's entire builder.

xlai20 · 2026-06-22T06:56:55Z

@joshuatants I've made 9 more commits since your review, to address your concerns and also further improved the code. These are the changes after your review:

API Simplifications:
- Removed with_scheme() and with_bucket_bound_hostname() methods from PostPolicyV4Builder.
- Unified endpoint processing logic in a new resolve_endpoint() helper that standardizes the http/https prefix before resolving the host.
Bucket Name Validation: Added check_bucket_name() to strictly enforce that the provided bucket name must begin with the "projects/_/buckets/" prefix.
System Keys Protection: Extended the protected system_keys check to include "x-goog-signature" and "policy", guaranteeing that user-supplied malicious or conflicting fields are ignored in favor of the system-generated defaults.
Documentation & Doc Tests: Added comprehensive Rustdoc examples (/// # Example) and doc tests covering how to use PostPolicyV4Builder and each of its builder configuration methods.
Test Updates:
- Updated existing tests to accommodate the "projects/_/buckets/" prefix requirement and the unified endpoint behavior.
- Added a new post_policy_v4_custom_fields edge-case test to verify that custom system fields correctly drop/override conflicting user-supplied configurations while preserving standard custom fields (e.g., x-goog-meta-*).
Example Script Updates: Modified the example generate_signed_post_policy_v4.rs to format the bucket string properly with "projects/_/buckets/" and included an extra "x-goog-meta-test" metadata field demo.

joshuatants · 2026-06-23T01:58:24Z

+/// Creates [V4 Signed Policy Documents] (POST Object Forms).
+///
+/// This builder allows you to generate signed V4 POST policy documents for Google Cloud Storage.
+/// A [Signed Policy Document] enables unauthenticated users to upload files to GCS using an HTML form


I don't see a definition for Signed Policy Document. Does this link work?

joshuatants · 2026-06-23T02:07:48Z

+        if let Some(port) = url.port() {
+            host.push_str(&format!(":{port}"));
+        }


Perhaps confusingly, if an explicitly specified port is the default port for a given scheme,url::port will not return it (https://docs.rs/url/2.5.8/url/struct.Url.html#method.port), i.e.

https://host:443 -> host
http://host:80 -> host

Based on

google-cloud-rust/src/storage/src/storage/signed_url.rs

Lines 472 to 477 in bbfc163

// Extract host and port exactly as they appear in the endpoint.

// We do this because the url crate omits default ports (80/443),

// but GCS requires them to be maintained if explicitly provided.

let path = url.path();

let scheme = format!("{}://", url.scheme());

let host_with_port = endpoint.trim_start_matches(&scheme).trim_end_matches(path);

, we need to keep the default port if it's specified. Let's reuse the logic from SignedUrlBuilder, perhaps refactoring if it makes sense.

joshuatants · 2026-06-23T02:17:07Z

+            .with_expiration(Duration::from_secs(30 * 60)) // 30 minutes
+            .with_field("Content-Type", "text/plain")
+            .with_field("x-goog-meta-test", "data")
+            .with_starts_with("$key", "")


"" is the empty string, sowith_starts_with("$key", "") is trivially true.

xlai20 added 10 commits June 18, 2026 03:23

feat(storage): implement GCS V4 Signed Policy Document (POST Policy)

154b337

docs(storage): output HTML form in v4 post policy example

17d24ea

refactor(storage): restrict with_timestamp visibility to test only

152009f

Fix syntax error of integration test

8851aff

undo changes in googleapis#5890

a53e9df

Standardize the sample code

0a551a5

format

171a28a

Idiomatic Rust

3437078

fix build error

04e2702

Revert "undo changes in googleapis#5890"

6bd1367

This reverts commit eded1b1.

product-auto-label Bot added the api: storage Issues related to the Cloud Storage API. label Jun 18, 2026

gemini-code-assist Bot reviewed Jun 18, 2026

View reviewed changes

This was linked to issues Jun 18, 2026

storage_generate_signed_post_policy_v4 #3079

Open

Feature parity for generateSignedPostPolicyV4 #4746

Open

xlai20 added 4 commits June 18, 2026 03:42

Fix errors reported by gcb-pr-format

684123c

Fix based on Gemini Code Assist review feedback

ce501de

fix error of gcb-pr-crypto-providers

d974a3d

Fix gcb-pr-format error

2234840

xlai20 marked this pull request as ready for review June 18, 2026 07:44

xlai20 requested review from a team as code owners June 18, 2026 07:44

xlai20 requested a review from joshuatants June 18, 2026 07:44

xlai20 added 2 commits June 18, 2026 08:25

Add edge case unit test

fb2b294

format

e428d04

joshuatants requested changes Jun 18, 2026

View reviewed changes

xlai20 added 4 commits June 19, 2026 04:58

Remove with_scheme(), enforce tests for universe_domain and endpoint

a0b673c

Handle conflict btw user custom fields and system fields

51cd42d

bucket name prefix check for API consistency

5fac312

Remove bucket_bound_hostname API as it conflicts with endpoint API

bb9bfb1

xlai20 added 5 commits June 22, 2026 03:12

Fix bucket name in post policy sample

020659c

Make sample consistent with other language sample

0e9df07

Make the post_policy code cleaner

bb19e82

Add Doc Test to post_policy

1e930ab

Correct documentation link

50c0664

xlai20 requested a review from joshuatants June 22, 2026 06:52

joshuatants approved these changes Jun 23, 2026

View reviewed changes

	// Extract host and port exactly as they appear in the endpoint.
	// We do this because the url crate omits default ports (80/443),
	// but GCS requires them to be maintained if explicitly provided.
	let path = url.path();
	let scheme = format!("{}://", url.scheme());
	let host_with_port = endpoint.trim_start_matches(&scheme).trim_end_matches(path);

Conversation

xlai20 commented Jun 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Key Additions

Testing & Validation

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

codecov Bot commented Jun 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

Uh oh!

joshuatants Jun 18, 2026

Choose a reason for hiding this comment

Uh oh!

xlai20 Jun 22, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

xlai20 commented Jun 22, 2026

Uh oh!

joshuatants Jun 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

joshuatants Jun 23, 2026

Choose a reason for hiding this comment

Uh oh!

joshuatants Jun 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

xlai20 commented Jun 18, 2026 •

edited

Loading

codecov Bot commented Jun 18, 2026 •

edited

Loading

joshuatants Jun 23, 2026 •

edited

Loading

joshuatants Jun 23, 2026 •

edited

Loading