Offline Amazon S3

Question

So I would like to try using Heroku to be my server, however heroku does not allow writing to it's file system. Instead I need to use the database and something like Amazon S3 for storage of things like uploaded images.

The problem is that I often don't have internet access when developing. Or very poor internet access. So developing for Amazon S3 is kind of impractical. Is there an offline version to use so that my local machine can act as the S3 cloud, and when in testing/production environments I can use the real S3?

Answer 1

My recommendation is to try s3fs with rsync. Here's how it would work:

1. Mount your s3 drive to /mnt/sdaX/ on your production machine and /mnt/sdaY/ on your local machine.
2. Create a file system at /mnt/sdaX/ on your local machine.

Make the changes on your local machine as needed. When appropriate, rync /mnt/sdaX/ to /mnt/sdaY/ on your local box.

I realize that this is complicated, but I'm not sure that there's really any other way to do it while maintaining the same configuration in both places. Normally I'd say you should just write to the s3fs drive locally with local caching enabled, but I'm not sure what happens when you return online (I'm pretty sure it doesn't sync, but I've gone ahead and asked s3fs developers).

Best,

Zach

Developer, LongTail Video

Answer 2

Have a look at:

• Eucalyptus Walrus
• Park Place

It might be some work to get them running, however. I finally wanted to write my own clone using node.js, but it has moved far away from the original S3 API, so it won't really help you anymore.