Continuous integration is one of the best inventions in the last 10 years in software engineering. Maven, despite all the criticism it gets, is one of the best software building tools made in the last 10 years. GitHub and Git are among the best source code management and versioning (SCM) I’ve ever used. Recently I’ve had a chance to try all this stuff and I think it might be useful to share the details.
The Problem
![[Dilbert on Software Versions, courtesy of dilbert.com]](http://localhost:8080/wp-content/uploads/2013/04/version_dilbert.gif)
Here at the BioSample Database Project, we have two Maven projects, a core business model, and an extension of it, biosd_model (you can happily ignore what exactly they’re about and call them ‘foo’ and ‘foo-extension’). From time to time we need to push a button and see the following happening:
core_model release
- Build the package, as project.
- If 1. worked, assign a new version, let’s say 2.0, by changing the POM file.
- Tag the SCM repository for the package with the same version defined at the previous step. Here we’ll talk specifically about GitHub and Git.
- Change the POM again, by defining a development version, e.g., 2.1-SNAPSHOT.
- Deploy the new release 2.0 that was defined before (mvn deploy, site-deploy)
biosd_model dependency upgrade
- Change the POM dependency on generic_model, so that the dependant points to the new 2.0 release. Test the change.
- If everything is fine, check-in the dependency upgrade on GitHub.
After having spent a few hours with Google and documentation pages, I realised Maven has plug-ins for all the steps above and the whole sequence can be easily automated by using them from a Bamboo plan. Let’s see how.
Releasing Machinery
The Maven Release Plug-in does almost all about the release sequence above and that’s great. I have ‘mvn clean release:clean’ as a first task in Bamboo (after GitHub update tasks). As the documentation says, the ‘release:clean’ bit just removes intermediate files and alike, produced by previous invocations of other plug-in goals. Most of the time it fixes failed releases, releases launched in ‘dry-run’ mode and all that jazz. The problem is it cannot do it all. For instance, if a previous command tagged the GitHub repository (see below), you’ll need to remove it manually, using Git commands.
Most of the releasing job is made by a command like this:
mvn release:prepare --batch-mode -DignoreSnapshots=true
-DreleaseVersion=${bamboo.newVer} -DdevelopmentVersion=${bamboo.newDevVer}
-Dtag=v${bamboo.newVer}
This does all the things you should do when releasing a package. In short:
- Checks the local copy and the SCM are in synch. Without ‘-DignoreSnapshots=true’ also enforces the use of non-snapshot dependencies. This is often a wise thing to do, since stable releases should rely on stable dependencies. Hence, I don’t do it here… welcome back in the real world.
- Updates the POM with ${bamboo.newVer} (more on this in a moment)
- Run tests (the usual ‘mvn test’)
- If OK, commits and tags the SCM with v${bamboo.newVer}
- Changes the POM again, with the value ${bamboo.newDevVer} and commits again. So, at the end you leave the SCM with a new snapshot release, new developments will be associated to such new release, as it should be.
Things like ${bamboo.newVer} are supposed to be replaced by some input value (provided by the user at execution time) somehow. I’ve put commands like the above into a Bamboo Plan and Bamboo allows to define variables, which can have values assigned when you run the plan manually. I’ve also had the idea to define defaults like: ${bamboo.newVer} = ‘e.g., v1.0’. Of course the same approach can be used almost unchanged if you implement the same kind of automation via shell scripts (or any other scripting language).
Setting up the SCM and its access
You’ll need to fix a few more things related to GitHub to make this working. First, you need to tell Maven the SCM details. This can be done in the POM:
scm:git:git@github.com:EBIBioSamples/core_model.git
...
Next, you need to sort out the authentication to GitHub. You’ve two options. One is to use the HTTPS connection, combined with -Dusername=… and -Dpassword=… , two parameters available in the Release plug-in (which, I guess relies on the SCM plug-in). Well, it didn’t work for me, it would keep saying access denied.
So, I turned it to SSH access. If not already there, you need to generate an SSH key-pair and for the user that will run Bamboo plans (or equivalent). Then you have to tell GitHub the public key of the pair. GitHub guys explain this better than I could do. What I had initially unclear is that a given public key can be used with a GitHub user only, so if a given SSH user on your side has to release for multiple GitHub projects and users, you will need some trick to overcome such a limit. For instance, you can define a GitHub shared user and add it to all the repositories you need to access to.
Just to add a further nuisance, recent Maven releases have eliminated SSH support from the default shipping. I fail to understand how this can be considered optional, but at least I know how to fix the problem.
Let’s do it for real
If the above release:prepare command went fine, now you have to finalise it (i.e., to add another Bamboo task):
mvn release:perform --batch-mode
–batch-mode is quite important in a continuous-integration context, as you can imagine it means: don’t ask me anything, cause no human being will be there to answer. This step also runs the 5th step in the sequence introduced above, very nice.
Dependencies Update
Now that you have a shiny new release in place, you should go over all the POMs that refer to it (all you have and you want under control) and make them to point to that. OK, you could do this manually, but often it’s worth doing it automatically. I defined a separated Bamboo plan for this, which I trigger manually.
It has two steps:
mvn versions:set -DgroupId=uk.ac.ebi.fg -DartifactId=core_model
-DnewVersion=${bamboo.newVer} -DoldVersion=${bamboo.oldVer}
This uses the versions plug-in to change the dependencies in the POM. Should your project be multi-module, the documentation says that children are changed too (well, you should use dependency management on the top POM anyway). Of course ${bamboo.newVer} are plan variables again.
Next, you have to send such change to the SCM:
mvn scm:checkin
-Dmessage='Updating core_model dependencies to ${bamboo.newVer} version [Bamboo auto-generated].'
This is the SCM plug-in in action. Never forget to record in the SCM what you’re committing, with proper comments, and no, versions:commit has nothing to do with this kind of action, it’s only used for a sort of internal two-step procedure designed for the versions plug-in. I usually just ignore it.
The End
Well, that’s all folks. I’ve done all of the above in about two days, first to crawl Internet for documentation and try the several Maven plug-ins from the command line. Second day to set-up Bamboo and check it all works. It’s time spent very well: now I can release and update dependencies in a matter of minutes.
So, happy hacking and please let me know your way of arranging tasks similar to the ones described here.
Click to rate this post!
[Total: 0 Average: 0]
