views:

138

answers:

4

I'm currently quite troubled by the way of deployment my team is adopting... It's very old-fashioned and I know it doesn't work very well. But I don't exactly know how to change it, so please give some suggestions about it...

Here is our current setup:

  • 2 webservers
  • 1 database server
  • 1 test server

Current deployment adaptation

  1. We develop and work on the test server, every changes is uploaded manually to the test server.
  2. When a change or feature is complete, we then commit the changes to SVN repository.
  3. After committing the changes, we then upload our changes to the first webserver, where there will be a cronjob running every minute to sync the files between the servers.

Something very annoying is, whenever we upload a file just as the syncing job starts, the file that is sync-ed will appear corrupted, since it is only half-uploaded. Another thing is whenever there is a deployment fault, it will be extremely difficult to revert. These are basically the problem I'm facing, what should I do?

In addition, since there are files on the 1st webserver which needs to be sync-ed to the other servers anytime, so the cronjob was there for the reason.

P/S: I'm sorry I forgot to mention that, the SVN server is hosted. We don't have too much control over it, but I believe I can edit hooks...

A: 

As a partial fix, create a URL on the web site that triggers the rsync command when you visit it, so that you can control when rsync runs (obviously, you need to make sure to hit server 1 with this URL and not expose this URL to the public). Better still, use curl to invoke the rsync URL at the end of whatever upload script you use.

Marcelo Cantos
I think I forgot to add something important, we have files on the webservers that might be changed at any minute and needs syncing as well, so unfortunately this solution can't be applied. I will add it to my question.
TheOnly92
Just use two different rsyncs for the two types of content.
Marcelo Cantos
@TheOnly92: if the application is updating files on the local file system of the individual server, and the servers are load-balanced, you have more to worry about than just deploying the application. You might want to think about shared storage for these files - either a shared filesystem or database backend.
Andy Shellam
Good point @Andy. This assumes, of course, that the frequently-updated content comes from the web server itself. I assumed (perhaps incorrectly) that it came from some back-end data feed.
Marcelo Cantos
+2  A: 

Check out svn export it produces a clean structure without the .svn directories

http://svnbook.red-bean.com/en/1.0/re10.html

Lizard
subversion is actually not installed on the webservers... but would that help on deployment?
TheOnly92
Deffinately in my opinion
Lizard
If so, I will need to login to the server via SSH each time a new feature is introduced, it wouldn't be of any problem if it's a major feature, but what if it was only a small bug fix? Redeploying everything again?
TheOnly92
why you don't like `.svn` directories?
zerkms
.svn directories are a vulnerability...
TheOnly92
Block access to them. If you're using Apache, the sample configuration already blocks access to any file/directory called ".ht*", so use the same technique to block any request that contains ".svn". I'm sure IIS could do the same on Windows.
Andy Shellam
Just noticed this discussion popup on the SO latest list: http://stackoverflow.com/questions/175056/svn-checkout-or-export-for-production-environment
Andy Shellam
+5  A: 

Use a deployment framework like Phing to manage deployment to the webservers and get rid of the cron job. Basically, a release on the production system should not happen automatically, but only after you are certain the current build is not broken. And it should not have a dependency on the dev system.

Since Phing uses XML and PHP to configure and control the deployment process, you can version control the process. This is an added benefit, as you can then keep the deployment connected to specific builds of your application.

To prevent the production website from being affected by the deployment process, consider uploading a new build into a separate directory and then just symlink to it. If anything goes wrong, you can easily symlink to a previous version.

Also consider using a CI server.

Gordon
The problem with CI servers is that I hate interacting with Java on Linux, a lot of problems for me. But Xinc appears to have stopped development...
TheOnly92
@TheOnly92 check out Slide 28 for available alternatives. Arbit for instance is written entirely in PHP.
Gordon
@Gordon Arbit seems to be in early development, is it reliable?
TheOnly92
We use Capistrano (Ruby app) for deploying our PHP site to all stages, which is very similar to what @Gordon is talking about (minus the CI stuff). Works beautifully.
Bryan M.
@TheOnly92 don't know how reliable it is right now. But at least it's actively developed and I know for sure it won't die anytime soon, since the folks developing it are friends of the guys who maintain most of these http://phpqatools.org/
Gordon
Does Capistrano needs to be run on the webserver side or could it be on the test server side? Can it directly deploy to the webservers then?
TheOnly92
+1  A: 

I did the same thing at my last place. What we had was:

  • a repository per website
  • a branch for each website dedicated to the live site (i.e. /branches/live) and test site (i.e. /branches/testing)
  • 2 webservers which could talk to SVN
  • a test server which could talk to SVN
  • all servers had SVN command-line client installed

Each webserver operated independently, so they didn't really know about each other - that was left to the load balancer. Each server had a cronjob that ran every 3 hours and exported the latest version of each website's live branch into the correct folder on the file system.

On our test server, it was a checkout of the testing branch for each website, and no cronjob. Developers updated the folders whenever they wanted to put something out for users to test before it went live.

During developments, commits were made to the trunk of the website. When changes were ready for testing, they were merged into the testing branch and the checkout updated manually on the test server. When changes were ready to go live, they were merged to the live branch and the servers would have updated by the end of the day.

In 3 years we only had one issue where a developer had committed something incorrectly and had to roll the site back.

Andy Shellam
We need the webservers to communicate to each other... User uploaded images needs to be sync-ed across and we have several other files that needs to be maintained the same.
TheOnly92
As I pointed in another comment, that's a really bad idea. You need to find a common backing store (shared filesystem or database) for the user-supplied content. What are the "several other files that need to be maintained the same" - are they user-supplied? If so, they also need to go in a common store. Otherwise you could maintain those in SVN.
Andy Shellam
The "other" files are translated .po files, they can't be stored in the SVN as it will take up so damn lots of space (oh yeah, btw, our SVN server is hosted). That's the biggest trouble.
TheOnly92
How much space are you talking, and how often (and how much) do they change? SVN only stores deltas of changes, so if the first commit of the files take up 200MB, but the files only change a few bytes each commit, it'll take you thousands and thousands of commits to even double that.
Andy Shellam
1 of our .po files can take up to 70+ MB, and we don't have only 1 translation, but more than 50... There are maybe more than 100 changes everyday so we can't take them into SVN.
TheOnly92