S3: Attach the Context to the Object, Not a Sidecar

The object was always the blob. The meaning lived somewhere else. Labels in DynamoDB. Lineage in a sidecar JSON next to the parquet. Classification in a spreadsheet the auditor asked you to export. Every pipeline that copied the file had to remember to copy the join.

S3 annotations, announced June 16, 2026, attach that context to the object version. Up to 1,000 named payloads, each up to 1 MiB, totaling 1 GiB. JSON, XML, YAML, or plain text. You put, get, list, and delete them without rewriting the object. That is the useful change. The launch copy calls them queryable knowledge bases. The engineering change is you no longer maintain a second store whose primary key is the S3 URI.

What you already had

S3 already describes objects three ways. None of them holds a training-label JSON or a processing history.

Tags stay the control plane. IAM, Lifecycle, and Cost Explorer still read tags, not annotations. User-defined metadata still requires a rewrite of the object. Annotations fill the gap those two left: a payload large enough for JSON, cheap enough to update, attached to the same version as the bytes.

You write them after the upload

PutObject and multipart upload cannot carry an annotation. The object lands first. Then you call PutObjectAnnotation. Bulk work is S3 Batch Operations invoking a Lambda that issues that call per key.

aws s3api put-object-annotation \
  --bucket amzn-s3-demo-bucket \
  --key datasets/training-v3.parquet \
  --annotation-name ml.labels \
  --annotation-payload /tmp/labels.json

The name is 1 to 512 bytes. Letters, digits, underscore, period, hyphen. It cannot start with aws or s3. The payload is UTF-8, 1 byte to 1 MiB. Binary goes in as Base64. Encryption follows the parent object: SSE-S3, SSE-KMS, or DSSE-KMS with the same key. SSE-C objects cannot take annotations. Adding or deleting an annotation does not change the object's ETag. Use --object-if-match with that ETag when you need to refuse a write against a version that has already been overwritten.

A new object version does not inherit annotations from the previous one. Overwriting the object replaces the annotation set with whatever the new version has, which is nothing until you write them again. Annotation deletes are permanent even in a versioned bucket. There is no delete marker for an annotation name.

Directory buckets, S3 on Outposts, File Gateway, Storage Lens, and Inventory reports do not see annotations. Design around that before you treat the feature as the system of record for every tool that already lists the bucket.

Query is a metadata table, not a GET of the file

GetObjectAnnotation and ListObjectAnnotations read the payload without downloading the object. That call works on every storage class, including Glacier Flexible Retrieval and Glacier Deep Archive, without a restore. You skip object retrieval charges. You still pay the annotation request. Annotation storage is billed at S3 Standard rates even when the parent object sits in Glacier. A 1 GiB annotation set on a Deep Archive object is a Standard storage line, not a Deep Archive line.

Scanning thousands of objects for a label used to mean a GET of each object, or of a sidecar, plus the egress of whatever came back. The annotation APIs remove that GET. They do not remove a request charge. Price the annotation GET against the old object GET, not against zero.

At bucket scale you enable an annotation table on the S3 Metadata configuration. S3 writes a managed Apache Iceberg table, one row per annotation per object version, with the payload in text_value. You query it with Athena or any Iceberg engine. The journal table that tracks bucket events updates in near real time. The annotation table refreshes within about an hour. Enabling it backfills existing annotations and that backfill is billed.

SELECT bucket, object_key, name, text_value
FROM annotation
WHERE name = 'ml.labels'
  AND json_extract_scalar(text_value, '$.split') = 'train'

That is the discovery path for a pipeline that used to list prefixes and open files. Athena reads the table. It does not restore the parquet.

Copy follows. Replication almost follows.

CopyObject for objects under 5 GiB copies annotations by default. Set x-amz-annotation-directive: EXCLUDE when the destination should be the bytes only. Multipart copies, including aws s3 cp once the object is larger than about 8 MB, do not copy annotations unless you pass --copy-props all. Until the last annotation write finishes, the destination key exists without its context. Time jobs against that window if a reader can land on the new key immediately.

Live replication and Batch Replication copy annotation creates and updates with the object. Grant the replication role s3:GetObjectVersionAnnotationForReplication on the source. Each replicated annotation is a PUT at the annotation request rate. Deny s3:ReplicateObjectAnnotation on that role if the replica should stay a byte copy.

Annotation deletes on the source are not replicated. The destination keeps the name. Overwrites of the same name do replicate, so an update stays in sync and a delete does not. Recreate the same logical record under a new name on the source and the replica accumulates both. That is how a destination object walks toward the 1,000-annotation cap while the source looks clean. If deletes are part of the workflow, delete on both sides, or stop treating the replica as an identical metadata image.

Put labels and lineage here. Put the gate on tags.

Training labels, model version, feature notes, upstream keys, transformation records, review status, classification, policy notes: those are annotation payloads. They move with CopyObject. They survive a Glacier transition without a restore to read them. An auditor queries text_value. A training job filters the annotation table instead of joining a DynamoDB table that drifted last Tuesday.

Access control, Lifecycle transitions, and cost allocation stay on object tags. An annotation named Environment will not satisfy an IAM aws:ResourceTag condition, will not match a Lifecycle filter, and will not show up in Cost Explorer. Use three required tags for those gates, then put the rest of the story on the object.

The join you have been maintaining is the pattern this removes. The controls you already attached to tags are the pattern this does not replace.