YML Continuous integration and support infrastructure

Continuous integration (CI) is a software development practice.  It recognizes that while developers need to work in controlled ‘sandboxes’ to maintain velocity on features and bugs, frequent integration of their efforts is necessary to avoid end-of-project “integration hell.” This is the point where their work is suddenly found to be incompatible. These incompatibilities can range from minor source code conflicts that “break the build” to major architectural conflicts that have frightening ripple effects and may require massive design and code changes.  If CI is universally regarded as a “good thing”, why don’t all software development efforts adopt it?


Continuous integration requires an organization to clearly define a workflow and an associated tooling that can be consistently applied day-in and day-out. What triggers CI? How is success or failure detected and quantified? Who are the stakeholders and what kind of visibility do they need in the process? Once workflow decisions have been made, attention can then be directed to tools that support the workflow.


Here at Y Media Labs, we have a multitude of ongoing Android projects, each with its own design, engineering, and management resources. Members of these projects may or may not be in the same geographic location. Through our experience we have developed our own CI strategy. In its most basic form, a CI workflow consists of checkins to a source code repository, automated builds from the repository, and of build verification.


Source control for each project is hosted in the cloud by BitBucket, so it seemed logical to host the remaining CI tools in the cloud as well. Both permit access from anywhere in the world, and eliminate local network management and firewall considerations. Cloud-based services like BitBucket and GitHub provide a variety of commit notification mechanisms that can be employed to drive CI builds.


At the other end of the workflow, verification is provided by unit testing the CI build. One of the recurring challenges for Android application development is the ever expanding universe of Android devices. Y Media Labs owns a large sampling of these devices, but in order to permit more comprehensive device coverage, we use testdroid.com to test across a wider range of devices. Testdroid supports most of the popular Android test frameworks. Of particular interest is their extended support for Robotium. They provide an Eclipse IDE plugin that monitors user interactions with an Android device or emulator and generates the corresponding Robotium instrumentation.


A CI server sits between the systems used for source control and the build testing infrastructure. We are using the very popular Jenkins CI server to manage our CI workflow. Each project defines Jenkins jobs that checkout the relevant source code from the source code repository, builds and packages it, and finally submits it to Testdroid for verification. Once remote testing on Testdroid is complete, the results can be viewed on the Jenkins web interface.


Often times when it comes to software development, the devil is in the details. In this case, details like whether all checkins trigger a CI build, or whether only checkins to a specific branch do, may or may not be well covered by the hooks and plugins provided by the source code control system and the CI server. In particular, we found that our desired workflow did not have out-of-the-box support by BitBucket and Jenkins. We use Git on BitBucket and use feature branches and pull requests as part of the development workflow. However, we only want to trigger CI builds when feature branches are merged into the master branch on BitBucket.


To support this particular workflow and provide a framework for future workflow refinements, we developed a proxy that interprets BitBucket ‘post hook’ notifications and triggers a Jenkins CI job when it is appropriate to do so. The diagram below shows resulting Y Media Labs’ CI infrastructure.