Indigio Blog - Colorado’s premier Interactive Agency blog - specializing in SEO, Creative Design, Development and Analytics » Blog Archive » Easier Production Releases
Set up an external dns entry for example.com and www.example.com (and the appropriate firewall rules to get port 80 of that IP to this apache box) and you now have a website that load balances over two separate clusters. You’re ready for a release during BUSINESS HOURS!
Choose a cluster to release to first and comment out all of its servers in the web-prod balancer. Run an apachectl graceful and you should then have no traffic going to that cluster. Shutdown the cluster, do your release thing, start up the cluster and then send QA off to test the cluster using the specific prod-a or prod-b.example.com url. Once QA confirms it’s good, uncomment the cluster from the web-prod balancer and comment out the other one. Fire off another graceful restart and repeat the release procedure on that cluster.
And *poof* you just rolled out a release with no downtime and very little impact to end users! Of course, this was just a very simple example. You can get much more detailed in your cluster design, especially with JBoss and different components. Just remember your individual clusters have to be completely separate. They can’t share any resources. Although, in all my experience they’ve always shared a database. You just need to make certain that any database updates the devs require you to make for the upgrade won’t cause the older release any harm.
This concept should work in most situations, just could take more planning/design/set up. I’ve got a similar set up running with 36 JBoss servers, 18 in each cluster. Quite a bit more had to be configured, especially to make sure that each individual cluster acts independently, and then there’s the SSL issues and sticky session requirements. But it’s so worth it though, and everyone will love you for cutting out the need for past midnight release “parties”.
I should set this up!