S3 API

hybrids3 accepts AWS Signature Version 4 requests, so existing boto3 or AWS SDK code works after you change the endpoint and the credentials. The S3 operations map onto the same routes as the HTTP API; requests that carry SigV4 auth or x-amz-* headers get S3 XML back instead of JSON.

Authentication

SDK setting Value
endpoint_url The server URL, including any path_prefix, such as http://localhost:8080 or https://example.com/storage
aws_access_key_id The bucket’s public_key, or master_public_key
aws_secret_access_key The bucket’s key, or master_key
region_name Anything. SDKs require one and sign with it; us-east-1 is the usual choice
signature version s3v4

The SDK sends the access key ID in the clear inside Credential= and an HMAC in Signature=. hybrids3 looks the bucket up by that public_key, recomputes the signature with the bucket’s private key and compares the two. The private key never crosses the network. The verifier is app/sigv4.py.

Authorization: AWS4-HMAC-SHA256 Credential=uploads-id/20260101/us-east-1/s3/aws4_request, SignedHeaders=host;x-amz-date, Signature=<hmac>

A bucket’s credentials work only on that bucket. The master credentials work on all of them.

boto3

import boto3
from botocore.config import Config

# limited to the "uploads" bucket
s3 = boto3.client(
    "s3",
    endpoint_url="http://localhost:8080",
    aws_access_key_id="uploads-id",
    aws_secret_access_key="uploads-secret",
    region_name="us-east-1",
    config=Config(signature_version="s3v4"),
)

# every bucket, and the only client that lists them all
s3_admin = boto3.client(
    "s3",
    endpoint_url="http://localhost:8080",
    aws_access_key_id="master",
    aws_secret_access_key="your-master-key",
    region_name="us-east-1",
    config=Config(signature_version="s3v4"),
)

s3.put_object(Bucket="uploads", Key="file.txt", Body=b"hello")
s3.get_object(Bucket="uploads", Key="file.txt")["Body"].read()
s3.head_object(Bucket="uploads", Key="file.txt")
s3.list_objects_v2(Bucket="uploads", Prefix="images/")
s3.delete_object(Bucket="uploads", Key="file.txt")
s3.list_buckets()        # only "uploads"
s3_admin.list_buckets()  # every configured bucket

aws-cli

export AWS_ACCESS_KEY_ID=uploads-id
export AWS_SECRET_ACCESS_KEY=uploads-secret
export AWS_DEFAULT_REGION=us-east-1

aws --endpoint-url http://localhost:8080 \
  s3api put-object --bucket uploads --key file.txt --body ./file.txt

aws --endpoint-url http://localhost:8080 \
  s3api list-objects-v2 --bucket uploads

Stick to single-request s3api calls. The high-level aws s3 cp switches to multipart uploads for large files, which hybrids3 does not support.

Supported operations

S3 operation Notes
PutObject Single request only. max_file_size applies
GetObject  
HeadObject  
DeleteObject Succeeds even if the object did not exist
ListObjectsV2 Prefix and MaxKeys (up to 1000). See listing limits
ListBuckets All buckets with the master key, one with a bucket key. CreationDate is the time of the request
HeadBucket  
CreateBucket Does nothing. Succeeds for a configured bucket, 404 otherwise
GetBucketLocation Always the default region
Presigned GetObject and PutObject Generated by the SDK or by hybrids3. See Presigned URLs

Listing limits

Listings return at most 1000 keys, sorted, with IsTruncated always false and no continuation token. Delimiter is ignored, so there are no CommonPrefixes; every key under the prefix comes back flat. With more than 1000 matching objects, list with narrower prefixes.

What is missing

  • Multipart upload. SDK helpers that upload in parts, such as boto3’s upload_file above its multipart threshold, fail. Use put_object or lower the threshold past your largest file.
  • CopyObject. Do not call it. hybrids3 treats the request as a plain PUT with an empty body, so the destination object is replaced by an empty file.
  • DeleteObjects (batch delete) gets 405.
  • Versioning, ACLs, bucket policies, tagging, object lock, lifecycle rules, CORS configuration and server-side encryption.
  • Creating or deleting buckets. Buckets come only from the configuration.

Request checksums are not verified. hybrids3 signs over the x-amz-content-sha256 value the client sends but does not compare it with the body; see Security.