Use rsyncrypto to encrypt files from your plaintext directory to your encrypted directory, and decrypt files from your encrypted directory and your plaintext directory, using keys that you keep locally.
Use your favorite version control system (or any other version control system -- svn, git, mercurial, whatever) to synchronize between your encrypted directory and the remote host.
The rsyncrypto implementation you can download now from Sourceforge not only handles changes in bytes, but also insertions and deletions.
It implements an approach very similar to the approach that that "The Rook" mentions.
Single-byte insertions, deletions, and changes in a plaintext file typically cause rsyncrypto to make a few K of completely different encrypted text around the corresponding point in the encrypted version of that file.
Chris Thornton points out that ssh is one good solution; rsyncrypto is a very different solution. The tradeoff is:
- using rsyncrypto requires transferring a few K for each "trivial" change rather than the half-a-K it would take using ssh (or on a unencrypted system). So ssh is slightly faster and requires slightly less "diff" storage than rsyncrypto.
- using ssh and a standard VCS requires the server to (at least temporarily) have the keys and decrypt the files. With rsyncrypto, all encryption keys never leave the local computer. So rsyncrypto is slightly more secure.