A long time ago, my housemate (who is one of the only people who reads this – Hi, Kurt!) had this great explanation for why his homework wouldn’t compile: “It was perfect, so I fixed it.” Man, that so describes every programmer I’ve ever worked with. (Incidentally and orthogonal to the point of this post, I suspect that this attribute of programmers coupled with the increase in the global number of active programmers accounts for so much of the frustration I experience with software nowadays.)
So, I glued together a tool chain for building the band music binder for the Silicon Valley Pipe Band. We save the music as ABC files in a repo on GitHub; a TeamCity server in EC2 pulls the changes and builds the PDF files, then uploads them to our folder on Box; and GitHub and TeamCity notify our Slack group of the activity. Which is all a great example of using existing tools to do the job. I got it up and running quickly with a minimum of debugging and it just works. So, it’s perfect. And now I want to fix it.
For starters, TeamCity is freaking huge. It can’t run on a free-tier EC2 instance, and it requires a database as well. So, I have to pay Amazon about 20 bucks a month. Not a huge deal, but the band only gets money from performances (and donations) and so this amounts to a $240/year donation to the band. Sure, I can swing it, but if I get hit by a bus, who’s to say that someone else will pick up that slack?
Meanwhile, I just noticed that TeamCity has an upgrade. Trying to install the update fails because the instance I’m using doesn’t have enough free disk space. Golly! 1.03 gigabytes of update!
Clearly, TeamCity has a huge suite of capabilities that we’re not using. We don’t need to run tests, we don’t need a federation of worker bees grinding away doing continuous integration, and we don’t need all the whiz-bang plugin capabilities. We just need to handle a push notification from GitHub, pull master/HEAD and run a couple of shell scripts. I bet that can all be accomplished in way less than a gigabyte of object code. Something tells me that a few kilobytes of Perl and an NGINX process could handle this.
Could your code be open source? https://circleci.com/open-source/
Well, it sort of is. Thing is, we’re not actually compiling code. We are taking ABC files (text notation for music) and use the (open source) tool abcm2ps to convert them to PostScript. Then we use a different (open source) tool to convert the PostScript to PDF. Then we copy the PDF files to a folder on Box.com (not open source, but there’s a nice open source tool to mount that folder as DAV). The Slack and GitHub integrations are just webhooks and can be handled with curl and NGINX, respectively.
The big reason that out-of-the-box CI solutions don’t work is that abcm2ps is not a standard compiler and these CI services don’t like to let you define your own compilers. Part of my adventure in constructing our existing tool chain was building an RPM of abcm2ps that wasn’t broken for bagpipe music. Now, at least, I’ve got an AMI that works.
While it might be a buncha work: https://circleci.com/docs/2.0/custom-images/