Amazon S3cmd crashes on some large uploads

Question

I routinely upload large bzipped sql files to S3 and have been noticing it crashing lately with this error. What might be causing this? Its always the same files that crash, but I am able to upload larger ones without a problem so it doesnt seem to be a size limit.

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
    An unexpected error has occurred.
  Please report the following lines to:
  [email protected]
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

Traceback (most recent call last):
  File "/usr/bin/s3cmd", line 1040, in <module>
    main()
  File "/usr/bin/s3cmd", line 1020, in main
    cmd_func(args)
  File "/usr/bin/s3cmd", line 188, in cmd_object_put
    response = s3.object_put_uri(real_filename, uri_final, extra_headers)
  File "/usr/lib/python2.5/site-packages/S3/S3.py", line 195, in object_put_uri
    return self.object_put(filename, uri.bucket(), uri.object(), extra_headers)
  File "/usr/lib/python2.5/site-packages/S3/S3.py", line 175, in object_put
    response = self.send_file(request, file)
  File "/usr/lib/python2.5/site-packages/S3/S3.py", line 384, in send_file
    debug("MD5 sums: computed=%s, received=%s" % (md5_computed, response["headers"]["etag"]))
KeyError: 'etag'

Answer 1

S3 did not return an ETag [entity tag] in its response. Irrespective of what cause the ETag to be missing in the response, your version of s3cmd did not expect the ETag to be absent and aborted.

As far as I can tell, this should no longer be an issue in the latest version, 0.9.9.91, of s3cmd:

## S3.py ##
...
# S3 from time to time doesn't send ETag back in a response :-(
# Force re-upload here.
if not response['headers'].has_key('etag'):
    response['headers']['etag'] = ''
...
debug("MD5 sums: computed=%s, received=%s" % (md5_computed, response["headers"]["etag"]))
...

Ensure that you are using version 0.9.9.91 of s3cmd.

Version 0.9.9.91 of s3cmd will retry the upload. In the eventuality that subsequent retries also fail for this particular file, alter S3.py to print out the full S3 response, or use a packet sniffer [tcpdump on linux, Wireshark on Windows] to intercept that same full S3 response. It may contain additional hints as to what is causing this behaviour.

Cheers, V.