Skip to content

feat(storage): Add full object checksum validation when resuming async bidi uploads#16138

Open
v-pratap wants to merge 4 commits into
googleapis:mainfrom
v-pratap:bidi-write-full-object-checksum
Open

feat(storage): Add full object checksum validation when resuming async bidi uploads#16138
v-pratap wants to merge 4 commits into
googleapis:mainfrom
v-pratap:bidi-write-full-object-checksum

Conversation

@v-pratap
Copy link
Copy Markdown
Contributor

@v-pratap v-pratap commented Jun 5, 2026

When taking over or resuming an async bi-directional upload (resumable or appendable) via QueryWriteStatus or MakeAppendableWriter, if the server provides persisted_data_checksums, seed the client-side CRC32C hash function from the acknowledged persisted state and forward the checksums directly to AsyncWriterConnectionImpl. This enables complete end-to-end full object integrity verification upon finalization.

@v-pratap v-pratap requested review from a team as code owners June 5, 2026 07:35
@product-auto-label product-auto-label Bot added the api: storage Issues related to the Cloud Storage API. label Jun 5, 2026
Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request adds support for resuming unbuffered uploads with persisted data checksums in the asynchronous GCS client. When resuming, if CRC32C validation is enabled and the server provides persisted checksums, the hash function is initialized with the existing CRC32C value and size. Feedback highlights a critical bug in MakeAppendableWriter where a fallback to CreateNullHashFunction is needed when persisted_size > 0 but validation cannot be seeded, preventing incorrect checksum calculations. Additionally, the new unit test should be enhanced to explicitly assert that checksums are not sent when validation is disabled.

Comment thread google/cloud/storage/internal/async/connection_impl.cc Outdated
@v-pratap v-pratap requested a review from kalragauri June 5, 2026 08:10
@v-pratap v-pratap force-pushed the bidi-write-full-object-checksum branch from cdb9a70 to 10e29d0 Compare June 5, 2026 08:13
@codecov
Copy link
Copy Markdown

codecov Bot commented Jun 5, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 92.71%. Comparing base (8d79f18) to head (d2dec7f).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main   #16138      +/-   ##
==========================================
- Coverage   92.72%   92.71%   -0.01%     
==========================================
  Files        2356     2356              
  Lines      220517   220617     +100     
==========================================
+ Hits       204468   204543      +75     
- Misses      16049    16074      +25     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

api: storage Issues related to the Cloud Storage API.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant