perf(storage): remove unnecessary mutex from multipart upload#2393
perf(storage): remove unnecessary mutex from multipart upload#2393ValentaTomas wants to merge 1 commit intomainfrom
Conversation
- Remove per-part MD5 computation (Content-MD5 is optional for GCS XML multipart uploads and the final object has no MD5 hash anyway) - Configure HTTP/2 transport with connection pooling and larger buffers - Remove unnecessary mutex on parts array (goroutines write to distinct indices)
PR SummaryMedium Risk Overview Reviewed by Cursor Bugbot for commit c4f0100. Bugbot is set up for automated code reviews on this repo. Configure here. |
| g.SetLimit(maxConcurrency) // Limit concurrent goroutines | ||
|
|
||
| // Thread-safe map to collect parts | ||
| var partsMu sync.Mutex |
There was a problem hiding this comment.
You're removing the mutex because each slot in the array is modified exactly once, and never read until it is returned, right?
There was a problem hiding this comment.
LGTM — removing optional Content-MD5 and the now-unnecessary mutex are both correct optimizations.
Extended reasoning...
Overview
Single-file change to packages/shared/pkg/storage/gcp_multipart.go. Removes MD5 per-part computation and a mutex guarding the parts slice in the concurrent upload loop.
Security risks
None. MD5 here was used only as an optional data-integrity hint to GCS, not for authentication or authorization. Removing it has no security implications.
Level of scrutiny
Low. Both removals are mechanical and independently verifiable: (1) GCS XML API documents Content-MD5 as optional; (2) each goroutine writes to a distinct slice index (parts[partNumber-1]), so no two goroutines can race on the same element, and errgroup.Wait() provides the required happens-before barrier. The -race test run is a strong signal.
Other factors
No interface or behavioral changes. Single file, no CODEOWNER concerns, no security-sensitive code paths touched.
|
We will reopen this when we merge compression as that will provide us a way to check integrity that makes this obsolete. |
Remove mutex on parts array — goroutines write to distinct indices. Supersedes #2381.