How to backup an AppEngine site?

Question

So, you build a great shiny cloudy 2.0 website on top of AppEngine, with thousands upon thousands of images saved into the datastore and gigs of data at the blobstore. How do you backup them?

Ok, so this is how to do it:

1/ Add this to the app.yaml (and update the app, off course):

- url: /remote_api
  script: $PYTHON_LIB/google/appengine/ext/remote_api/handler.py
  login: admin

2/ Run this command to generate the special file needed:

appcfg.py create_bulkloader_config --filename=bulkloader.yaml --url=http://appName.appspot.com/remote_api

3/ Check the bulkloader.yaml file looking for lines including TODOs, there will be at least one with the format of the backup file (xml, csv, ...).

4/ And finally, do the backup of each datastore entity:

appcfg.py download_data --config_file=bulkloader.yaml --filename=entityBackup.csv --kind=nameOfEntity  --url=http://appName.appspot.com/remote_api

Please note, download_data, NOT --dump as stated in the docs... and big thumbs down to the translation of the documentation, because the English version is much better!

Answer 1

use google app engine data export http://code.google.com/appengine/docs/python/tools/uploadingdata.html

Answer 2

The command here does not work

http://code.google.com/appengine/docs/python/tools/uploadingdata.html#Downloading_and_Uploading_All_Data

--dump needs to be replaced with download_data, --restore needs to be replaced with upload_data and --app_id needs to be changed to --application

Then it would read

Downloading and Uploading All Data

You can download and upload every entity of a kind in a format suitable for backup and restore, all without writing any additional code or configuration. To download all entities of all kinds, run the folowing command:

appcfg.py download_data --application=<app-id> --url=http://&lt;appname&gt;.appspot.com/remote_api --filename=<data-filename>

You can also use the --kind=... argument to download all entities of a specific kind:

appcfg.py download_data --application=<app-id> --kind=<kind> --url=http://&lt;appname&gt;.appspot.com/remote_api --filename=<data-filename>

Note: Downloading all entities of all kinds only works on App Engine, and does not work with the development server. To upload data to the app's datastore from a file created by appcfg.py --dump, run the following command:

appcfg.py upload_data --application=<app-id> --kind=<kind> --filename=<data-filename> <app-directory>

When data is downloaded, the entities are stored along with their original keys. When the data is restored, the original keys are used. If an entity exists in the datastore with the same key as an entity being restored, the entity in the datastore is replaced. You can use --restore to replace the data in the app from which it was dumped, or you can use it to upload the data to a different application. Entities with numeric system IDs will be restored with the same IDs, and reference properties will be preserved.