What backup strategy do you use for your code?

Question

Everyone uses source-code control to manage versions (right?) and this provides some level of backup. There are times, however, when your local copy is out of sync with the repository. Moreover, some sandbox-type projects may not have yet ;-) made it into SCC.

EDIT: I have multiple projects in my projects directories. Not all are in current development but anyone of which might need to be "fixed" whenever a bug is found. Restoring a single, active project from SCC seems perfectly reasonable. Restoring all of the couple of dozen projects that I support from SCC seems less reasonable than restoring from a backup and syncing as necessary from SCC.

What backup strategies, other than source code control, do you use to keep your code safe?

A similar question can be found at http://stackoverflow.com/questions/38388/organization-wide-backup-strategy, but I'm more interested in hearing others' personal strategies if you happen to work in an organization that has no overal strategy. I'll provide my strategy in an answer.

Answer 1

I use Microsoft SyncToy 2.0 to synchronize my project directories with a folder on a network share. I have separate scheduled tasks that run different SyncToy scripts for the various directories (broken down by Visual Studio version).

Answer 2

My strategy is always check in, and backup the entire repository.

I never leave anything out of source control and I make sure regular backups (incremental daily, full weekly and monthly rotation) are happening and are functional.

Answer 3

We also check everything in (check in early and often) and back up the entire repository (CVS) with tar and ftp it to our backup server.

Answer 4

(In addition to source control to a remote server) I use the free version of SyncBack (www.2brightsparks.com) and this batch file: (where the arguments to syncback.exe specify previously configured syncback backup profiles)

@echo off

echo Stop and start SQL Server
echo -------------------------

net stop "SQL Server (SQLEXPRESS)"
net stop "SQL Server (SQLSERVER2008)"
echo -----------------------------------------------------------
echo Back up running now... please wait.

"C:\Program Files\2BrightSparks\SyncBack\SyncBack.exe" c e-contents f-contents

echo Backing up done. Starting SQL Server...
echo -----------------------------------------------------------

net start "SQL Server (SQLEXPRESS)"
net start "SQL Server (SQLSERVER2008)"

echo -----------------------------------------------------------
echo Back up is done and SQL Server is running now.
echo -----------------------------------------------------------

pause

with two 8gb flash drives every day. At the end of the week, I do the same thing but then target a desktop external drive.

SyncBack is great!

Answer 5

In my opinion, rebuilding everything from SCC every now and then is anyway a good practice (during night time, for example). Doing so makes sure that you haven't forgotten to add any essential file to the repository. The whole procedure should anyway require at maximum a couple of steps.

Answer 6

When your writing something that doesn't (yet) belong in the main build, create a branch. When it should go into the main build, merge your branch with it.

Distributed VCSes also make local branches really easy, the central repository will never know they existed.

Backing up a local repository (of a distributed VCS) by pushing changes onto a remote copy is so trivial that I use git as my main method of backup for most documents, configuration files, basically anything non-binary.

Answer 7

For everything but the most simple 5 min test things I use version control, Subversion in my case.

I've used some old hardware which I run linux on and a Subversion server which I commit to. Then I have a cron script archiving the repository (if changed since last time) every night and attaching it in a mail to my gmail account with the changelog in the body. With the 20 mb attachment limit on Gmail all but the most binary intensive repositories may be backed up with splitting the files.

I plan to rework this to put the backups on Amazon S3 but haven't got around to do it yet.

Most important thing IMHO is to always have a backup someplace else (geographically), not just on a USB-drive or something.

In case of very small 5 min. tests I put them in my DropBox (www.getdropbox.com).

Answer 8

if you diverge from your source control for significant amounts of time, then you need some distributed source control.

Answer 9

Though this is a subjective answer, I think you are not using source control properly.

Yes, your local copy is often out-of-sync with the repository, but any given change should only be a small amount of work (eg, you shouldn't have stuff that is not checked in for days on end). If you are committing often, then in the case of a drive loss (theft/failure/etc) you lose a small amount (typically <1 day) of work.

If you are doing something totally crazy that is disruptive to other developers, then you should be working in a branch. When you're done, merge your changes back.

You should also be able to rebuild the project from your SCC system at any time. It's a good thing to do from time to time, just to make sure that everything you do need to build is in SCC - sometimes files get missed, and you never notice because you always build from the working copy that you've been using for the past 6 months.

Answer 10

I use Mercurial as my version control system. I use the repository on my windows laptop as my main repository but use Mercurial's clone feature to back it up to my ubuntu server every two or three days. I also use sync toy to back up important directorys to a flash drive including the copy of the repository found on my laptop.

Answer 11

At the end of the day I check my code into source control.

At around midnight Mozy kicks on and backs up my code off site.

At around 1AM the SC box gets backed up to tape.

At around 3AM Syncback SE wakes up and backs up my code to an external HD.

Throughout the day my work box syncs with my home box using Live Sync

Answer 12

I use Unison to replicate my entire home directory on two different machines at home. This way if I am sloppy or if I have 20-year-old files not under source control (.emacs) I still have a measure of protection. I also replicate everything except personal files (photos, music) on a machine at work as well.

Answer 13

OSX's Time Machine

Answer 14

Since I use TFS (Team Foundation Server) I just back up the SQL Server database as any other database I use

Answer 15

Version Control (SVN) is more than enough for me. Yet, there are some rules:

• I commit as often as possible (4-6 hours of work without committing already start creating this tingling sensation of something going wrong).
• SVN structures of solutions are always atomic. You just need a fresh CheckOut in order to be able to run "rebuild-copy-package" integration script on any solution (running tests might require providing DB connection settings before that).
• SVN server is reliable and backed up regularly.
• Changes are being propagated between the different solutions composing the application (i.e. from open source shared library to the internal code that leverages it) only via the commits (integration server picks this up and creates packages that could be used in the solutions down-stream).
• Sandbox projects (prototypes) are always kept in Prototypes folder of SVN (sibling of Trunk or Tags) being named "YYYY-MM-DD PrototypeName"

Answer 16

I'm working on a product called "Transactor Code Agent", which is designed to do just what you are asking for.

It provides local backup and version control for your source files.

It lets you use your existing source control setup for what it was meant for (managing "mostly completed" work by multiple developers over multiple releases), while providing you with an automated backup and local file version control for your in-progress work.

The beta should be coming out sometime in January.

You can see our "website" (it's a little rough) at

www.transactor.com

There's a form there you can use to signup for the private beta.

Update:

Here's a little bit more information, based on some feedback I got in the comments:

1) Do I have a thing againt source control?

No! I think source control is a wonderful thing. When used properly it provides a tremendous tool for managing the software life cycle.

But, when it is used properly, source control leaves a big gap, in that it doesn't protect a developers work until it's finished. What's needed is something that focuses on the in-progress work of individual programmers. Code Agent does that.

To put it differently, source control is a tool designed to make your boss's life easier (because it helps to manager features and changes and teams and versions over time)

Code Agent is a tool designed to make your life easier (because it makes sure that your work is always saved).

Answer 17

In case anything should fail, I will sometimes e-mail myself important pieces I am working on to webmail accounts such as yahoo or hotmail. I know everyone is talking about switching from paper to digital, but sometimes you never know what is going to happen, so I will print out hard copies. Obviously, this is not the best solution especially for a large project, so I will usually restrict hard copies to smaller, more important pieces. I also tend to be a bit paranoid, so I will end up taking a backup of a backup of a backup.

Answer 18

Code which isn't checked in (and hence backed up) to your VCS, does not exist. It's no more real than code which you just have in your head. It really is that simple.

Answer 19

DON'T USE BIDIRECTIONAL SYNC TOOLS FOR BACKUP

... well, at least not automatically

Sync tools, such as unison are synchronizing two (or more) locations. Thus, if you accidentally mess up a file in one location, the mess will be transformed to another one and you won't notice.

Answer 20

no one wants to have to rebuild their entire project directory from SCC if the disk drive dies

Huh? We always do it this way. In fact we have a build server that continuously performs fresh builds from a clean checkout. If restoring from a backup seems to be a better way than restoring from the SCC, you need to improve your SCC.

For all code that is not ready for production we have a directory called "playground" and "junk" in the SCC.

Answer 21

I use rdiff-backup to perform a daily incremental backup of my laptop over SSH. It uses deltra compression (like rsync) so it's very quick. It also lets you go back any number of days in the backed up data so that you can go back to right after you finished some complicated code, but before you accidentally deleted it all.

It's a little tricky to get going but well worth it in my opinion.

Answer 22

Aside from subversion, I use crashplan for online offsite backups. It can also backup to local storage and other computers (though unfortunately seems to require the same backup set to be stored at each destination currently - i.e. can't store a smallish set of vital stuff offsite, and a larger set locally.)

I also use unison (for things too large to backup offsite - music, movies etc), and an OSX time capsule, so that in the event of data loss, I can hopefully restore without resorting to online backup. The online backup is intended for a disaster like the house burning down or being burgled.

Answer 23

There are times, however, when your local copy is out of sync with the repository. Moreover, some sandbox-type projects may not have yet ;-) made it into SCC.

Firstly, you should really try to minimize the time your code is 'out' of SCC. Not for backup purposes, but for keeping track of what was done when, and particularly why. commit comments are invaluable. A big checkin containing 3000 files with the message "Initial Revision" is not very useful.

The sandbox projects argument does hold some weight, but then you should just treat it the same way you treat all your other files. Back them up to an external USB drive, or whatever. If you're not backing up all your other files, I suggest you start now.

And, of course, no one wants to have to rebuild their entire project directory from SCC if the disk drive dies -- much better to just restore from an actual backup.

Isn't this just svn checkout? Why not just 'rebuild from SCC' ?

Answer 24

Subversion: server is managed by Beanstalk, client uses Tortoise SVN. After every coding session, everything goes back to the SVN repository so I never have to worry about losing code. I also periodically back up the latest code to a CD and lock it in a vault just to be sure!

Also, keep in mind that your code is only one part of the equation. Most modern development environments require significant customization in and of themselves (integrating 3rd-party tools being the obvious example but just installing the IDE with appropriate options takes quite awhile as well). Thus, I also do all of my development in a virtual machine that I can easily back up to an external hard drive. These get locked in the vault as well.

Finally, I backup a "reference" database to complete the picture. I cannot just backup the schema because, in my product, there is significant system data kept in the database (e.g. content delivered with the web site).

Answer 25

I use MozyPro to automate an off site backup of the current code on my machine as well as the source code control database. This runs incrementally every night.

Answer 26

I use a product (that I wrote, it's my micro-isv), called Transactor Code Agent. Its a backup tool designed specifically for programmers.

It watches your source code, and every time you save a change it backs it up and keeps a local history for you.

I think it works a lot better for backup than source control does for several reasons:

1. Source Control is meant to be a change management tool, to help your program move from one consistent state to another
2. You don't have to worry about maintaing a private branch
3. You don't have to interrupt your work to make checks purely for backup purposes. You can just focus on writing code, and checkin your stuff when its done.

You can download a demo of it here:

http://www.transactor.com/download

Answer 27

Online (Internet) backups are an important part of the process.

All kind of backups to external drives are doomed to failure unless they are made by an appointed personal (such as a secreatry). If you're a very small shop (or a µ-ISV, such as me), this is not an option. Even then, where is the external drive kept? A safe with fire protection is the only possible good answer. Storing them offsite is not good: People WILL forget to bring it back to office for the periodical back-up.

Backups to NAS are IMHO a better solution than external drives. But the day the building is on fire, offsite backups are your only chance to stay alive.

I personnaly use Mozy to back up the main local directories in addition to the SCC DB.

Needless to say that AES-256 or similar encryption is a must have for storage of your source code on someone else's hard drives. Mozy and all its serious competitors offer it.

Answer 28

joel from joelonsoftware said in some post that if building and deploying your project takes more than two command lines (or more than one minute in preparations) you are doing it wrong. i completely agree with him, and i think a SCM should be enough. backups systems are just for catastrophic disasters (hdd failures, fire and tornados).