S3 API
hybrids3 accepts AWS Signature Version 4 requests, so existing boto3 or AWS SDK code works after you change the endpoint and the credentials. The S3 operations map onto the same routes as the HTTP API; requests that carry SigV4 auth or x-amz-* headers get S3 XML back instead of JSON.
Authentication
| SDK setting | Value |
|---|---|
endpoint_url | The server URL, including any path_prefix, such as http://localhost:8080 or https://example.com/storage |
aws_access_key_id | The bucket’s public_key, or master_public_key |
aws_secret_access_key | The bucket’s key, or master_key |
region_name | Anything. SDKs require one and sign with it; us-east-1 is the usual choice |
| signature version | s3v4 |
The SDK sends the access key ID in the clear inside Credential= and an HMAC in Signature=. hybrids3 looks the bucket up by that public_key, recomputes the signature with the bucket’s private key and compares the two. The private key never crosses the network. The verifier is app/sigv4.py.
Authorization: AWS4-HMAC-SHA256 Credential=uploads-id/20260101/us-east-1/s3/aws4_request, SignedHeaders=host;x-amz-date, Signature=<hmac>
A bucket’s credentials work only on that bucket. The master credentials work on all of them.
boto3
import boto3
from botocore.config import Config
# limited to the "uploads" bucket
s3 = boto3.client(
"s3",
endpoint_url="http://localhost:8080",
aws_access_key_id="uploads-id",
aws_secret_access_key="uploads-secret",
region_name="us-east-1",
config=Config(signature_version="s3v4"),
)
# every bucket, and the only client that lists them all
s3_admin = boto3.client(
"s3",
endpoint_url="http://localhost:8080",
aws_access_key_id="master",
aws_secret_access_key="your-master-key",
region_name="us-east-1",
config=Config(signature_version="s3v4"),
)
s3.put_object(Bucket="uploads", Key="file.txt", Body=b"hello")
s3.get_object(Bucket="uploads", Key="file.txt")["Body"].read()
s3.head_object(Bucket="uploads", Key="file.txt")
s3.list_objects_v2(Bucket="uploads", Prefix="images/")
s3.delete_object(Bucket="uploads", Key="file.txt")
s3.list_buckets() # only "uploads"
s3_admin.list_buckets() # every configured bucket
aws-cli
export AWS_ACCESS_KEY_ID=uploads-id
export AWS_SECRET_ACCESS_KEY=uploads-secret
export AWS_DEFAULT_REGION=us-east-1
aws --endpoint-url http://localhost:8080 \
s3api put-object --bucket uploads --key file.txt --body ./file.txt
aws --endpoint-url http://localhost:8080 \
s3api list-objects-v2 --bucket uploads
Stick to single-request s3api calls. The high-level aws s3 cp switches to multipart uploads for large files, which hybrids3 does not support.
Supported operations
| S3 operation | Notes |
|---|---|
PutObject | Single request only. max_file_size applies |
GetObject | |
HeadObject | |
DeleteObject | Succeeds even if the object did not exist |
ListObjectsV2 | Prefix and MaxKeys (up to 1000). See listing limits |
ListBuckets | All buckets with the master key, one with a bucket key. CreationDate is the time of the request |
HeadBucket | |
CreateBucket | Does nothing. Succeeds for a configured bucket, 404 otherwise |
GetBucketLocation | Always the default region |
Presigned GetObject and PutObject | Generated by the SDK or by hybrids3. See Presigned URLs |
Listing limits
Listings return at most 1000 keys, sorted, with IsTruncated always false and no continuation token. Delimiter is ignored, so there are no CommonPrefixes; every key under the prefix comes back flat. With more than 1000 matching objects, list with narrower prefixes.
What is missing
- Multipart upload. SDK helpers that upload in parts, such as boto3’s
upload_fileabove its multipart threshold, fail. Useput_objector lower the threshold past your largest file. CopyObject. Do not call it. hybrids3 treats the request as a plainPUTwith an empty body, so the destination object is replaced by an empty file.DeleteObjects(batch delete) gets405.- Versioning, ACLs, bucket policies, tagging, object lock, lifecycle rules, CORS configuration and server-side encryption.
- Creating or deleting buckets. Buckets come only from the configuration.
Request checksums are not verified. hybrids3 signs over the x-amz-content-sha256 value the client sends but does not compare it with the body; see Security.