Tuesday, March 15, 2011

Continuous integration and deployment using hudson and snapshot versions of artifacts

In my most recent project we are writing a rather large system that consists of at least a dozen components. We started off with true and agile...

big design upfront.

We sat down for a week with the clients and agreed on the interfaces... This seemed counterintuitive, especially in the perspective of agility and just doing the simplest thing that can possibly work. But it turned out to be quite useful. We started with a team of over 20 people that were working their assess off to achieve the goals of the sprint. They were working full speed from the start because of the head start that the interfaces definition meeting gave us. I cannot imagine agreeing on those interfaces as we go along. Especially if (as is in our case) the clients are not in the same office (and not in the same timezone for that matter). We would have never has any conclusive conversations and the thing would take forever to build. With the interfaces defined everybody could focus on the sprint goal. And the goal of the first sprint was:

an integrated system

To do that we relied on Hudson. The real cool feature of it is parsing the poms of the projects that it builds to find out the dependencies between them and to automatically construct build chains from there. So we ended up with a setup in which every commit to SVN triggers a shapshot deployment, which triggers downstream builds and deployments, which triggers building our webapp, deployment to tomcat and running functional tests on the deployed instance of the webapp. This is so insanely cool, that I want to shout out! Now, I know that most of the definitions of Continuous Integration are saying just that the build should be self-testing. They only talk about integration code, whereas I believe we should not only be integrating code (as in everything compiles and the unit/integration tests pass), but also a larger piece of the whole puzzle functionality (as in everything compiles + all the parts of a system we are building should compile and work together + all the funtional tests pass). This bit of integrating small pieces, to larger pieces, to yet larger pieces is what the continuous integration is all about. But every talk I've ever seen on this concentrated on integrating the code rather than functionality. Should I mention that pervasive testing is an absolute must to know whether any and all of those pieces work? Now, to be able to do functional tests you need to

continuously deploy

your stuff to the testing environment (be it a database instance, appserver instance, whatever) and run your functional tests there. In the maven case this means running everything off of a clean repository to eliminate issues with repo manager and things like that. This is of course simple with hudson (a.k.a Jenkins) + maven and the repo location setting.
The next part of the puzzle is

Releasing stuff.

This is a major struggle right now. In all the poms we have 0.1-SNAPSHOT dependency versions. This is cool for the continuous integration, but is a pain in the ass for releasing. There is a version plugin which can update the versions in poms to release versions, but in order to actually release the system, you need to create the release build chain yourself by hand. This is suboptimal, and I think that there must be a better solution to the problem. Does anybody have a solution?

No comments:

Post a Comment