Skip to content
Cloud Provider Benchmarks

Storage

Google Cloud Storage with S3 tools: HMAC keys, the XML API and what changes

How to point S3 tools and libraries at Google Cloud Storage with HMAC keys and the XML API, what gcloud can do with S3 buckets, and what to test before you rely on it.

Published: Updated: Türkçe oku

Direct answer

To use an S3 tool with Google Cloud Storage, point it at https://storage.googleapis.com, configure Cloud Storage HMAC keys, and test every S3 operation your application calls. The XML API is partially compatible and is not a full replacement for the S3 API.

Executive summary

Google Cloud Storage works with some S3 tools through its XML API. Google's interoperability guide says to change the request endpoint to https://storage.googleapis.com and to configure the tool with Cloud Storage HMAC keys. Requests can be signed with the V4 signing process. The gcloud CLI can also list and synchronise Amazon S3 buckets when S3 credentials are configured. Google's documentation describes it as interoperability with some tools and libraries, and the dataset classifies Google Cloud's S3 coverage as partial. This article covers setup and one documented limitation; it does not test feature parity.

Key findings

  1. 01 The XML API is the path for S3-style tools. Google describes it as interoperable with some tools and libraries that work with Amazon S3.
  2. 02 Authentication uses Cloud Storage HMAC keys with the V4 signing process. Google's guide also mentions RSA signatures.
  3. 03 gcloud storage ls and gcloud storage rsync accept s3:// URLs when Amazon S3 credentials are in the AWS credentials file.
  4. 04 Dotted S3 bucket names can fail in gcloud with an invalid certificate error. Google documents a path-style endpoint setting as the workaround.
  5. 05 Storage Transfer Service can import from Amazon S3 buckets, on a schedule or event-driven when S3 Event Notifications are used.

Setup for an S3 tool

  1. Obtain the Cloud Storage HMAC keys for the account that will access the buckets.
  2. Set the tool’s request endpoint to https://storage.googleapis.com.
  3. Configure the tool with the HMAC keys. Use the V4 signing process where the tool supports it.
  4. Run the operations your application uses against a test bucket before you point production traffic at it.

Using gcloud with S3 buckets

The gcloud CLI can work with Amazon S3 when S3 credentials are in the AWS credentials file. Google’s example lists the objects of an Amazon S3 bucket with gcloud storage ls s3://<bucket>, and synchronises an S3 bucket with a Cloud Storage bucket using gcloud storage rsync. The guide’s rsync example includes --delete-unmatched-destination-objects; check what that flag deletes before running a sync.

Dotted bucket names

Amazon S3 does not support virtual-hosted bucket URLs for bucket names that contain a dot. In that case gcloud can fail with an invalid certificate error. Google’s guide says to set the storage/s3_endpoint_url property to the regional path-style endpoint, https://s3.<region>.amazonaws.com, for the region that contains the bucket.

Moving data from Amazon S3

Storage Transfer Service can import data from Amazon S3 buckets into Cloud Storage. It can run recurring transfers, delete source objects after transfer, and select which objects to transfer. With S3 Event Notifications, it can keep a Cloud Storage bucket in sync with the source.

What to test

The S3-compatible object storage comparison lists the features that vary between providers. Test the ones your application uses:

  • Core operations: PUT, GET, DELETE and LIST
  • Multipart upload and presigned URLs
  • Versioning, lifecycle rules and object lock, if used
  • Bucket policies and event notifications, if used
  • The error codes your application handles

Methodology and limitations

  • Setup steps and gcloud behaviour come from Google’s interoperability guide, checked on 2026-11-12.
  • The article does not test feature parity or performance on Google Cloud Storage. For measured throughput, see the object storage benchmarks.
  • Google’s guide states that the XML API is interoperable with some tools; the feature list above is a checklist, not a result.

Conclusion

Google Cloud Storage can serve S3-style tools through the XML API with HMAC keys and an endpoint change. Treat it as partial compatibility, and test each operation your application depends on before moving data or traffic.

Frequently asked questions

Do I need a different SDK for Google Cloud Storage?

Not necessarily. Google's guide says to change the endpoint and configure HMAC keys for tools and libraries that work with Amazon S3. Test each operation, because interoperability is partial.

Can gcloud read an Amazon S3 bucket?

Yes. Google's documentation says gcloud storage can list objects in Amazon S3 buckets and synchronise them with Cloud Storage buckets once S3 credentials are added to the AWS credentials file.

Is XML API interoperability the same as S3 API parity?

No. The dataset records Google Cloud's S3 coverage as partial, and Google's guide describes interoperability with some tools rather than the whole S3 API.

Sources

  1. [1] Interoperability with other storage providers — Google Cloud Official documentation accessed: