Python Boto S3 to work with Custom Domains in Amazon S3

Question

How do I use the Python Boto library with S3 where the URL's it generate will be my CNAME'd subdomain to the Amazon S3 Server.

By default it uses the default format BUCKETNAME.s3.amazonaws.com but S3 supports custom domain aliasing using CNAME (so you can have custom.domain.com -> CNAME -> custom.domain.com.s3.amazonaws.com where "custom.domain.com" is the bucket. AWS Documentation

I see that the boto library has boto.s3.connection.SubdomainCallingFormat and class boto.s3.connection.VHostCallingFormat...

Anyone know how I can setup the boto.s3 where the generate URL's are for my own custom domain instead of the default?

Answer 1

1. Your CNAME records must be already pointing to your S3 bucket.
2. Your S3 bucket needs to also be named custom.domain.com
3. Verify you are able to access your files from custom.domain.com in your browser.

Once that's done, the following snippet I wrote will print the URL's to all the files within a key:

import sys
import boto.s3
from boto.s3.connection import VHostCallingFormat
from boto.s3.connection import S3Connection

def main():
    access_key = "<AWS_ACCESS_KEY>"
    secret_key = "<AWS_SECRET_KEY>"
    bucket = "custom.domain.com"

    # assuming you have your files organized with keys
    key_prefix = "css"
    key_prefix = key_prefix + "/"

    conn = S3Connection(access_key, secret_key, calling_format=VHostCallingFormat())
    bucket = conn.get_bucket(bucket)

    # get all the keys with the prefix 'css/' inside said bucket
    keys = bucket.get_all_keys(prefix=key_prefix)

    for k in keys:
        print k.generate_url(3600, query_auth=False, force_http=True)

    # output:
    # http://custom.domain.com/css/ie.css
    # http://custom.domain.com/css/print.css
    # http://custom.domain.com/css/screen.css
    # http://custom.domain.com/css/style.min.css

if __name__ == '__main__':
    main()