It’s not everyday that you find bugs in production code for world class providers like Amazon; however today I have found one of the easiest looked over bugs in the software world. An off by one error, this error is found in the Add Metadata section of AWS S3 and can be replicated via:
-
Login to AWS
-
Go to S3 Service
-
Go to Bucket
-
Actions > Upload
-
Add Multiple Files
-
Set Details >
-
Set Permissions >
-
Set Metadata >
-
Add More Metadata
-
Add a Key of Content-Encoding with the value gzip
-
Start Upload
From here your files will all be uploaded and all but the first file will have the custom Content-Encoding key; this leads me to assume that there is a simple off by one bug in the S3 upload console; however I am just speculating at this point (unfortunately amazon hasn’t open sourced their AWS console source code) or I’d try to find the actual bug even with the burning hatred of Java that I have.
Workaround: Until Amazon fixes this, you can apply metadata correctly to all files using the AWS CLI:
# Upload a single file with Content-Encoding metadata
aws s3 cp myfile.gz s3://my-bucket/myfile.gz \
--content-encoding gzip
# Apply to all files in a directory
aws s3 sync ./dist s3://my-bucket/ \
--content-encoding gzip \
--metadata-directive REPLACE
# Or update metadata on already-uploaded files
aws s3 cp s3://my-bucket/ s3://my-bucket/ \
--recursive \
--metadata-directive REPLACE \
--content-encoding gzip
Anyways just thought I’d share because I couldn’t find a way to report a bug through the console or a quick google search. I opened a ticket with them but we all know that’s going to take at least a few days for a response, so hopefully one of the many Amazon employees I know will see this post and submit the bug internally.
Tell your friends...