Efficiency in Development Workflows
DEVELOPING A NEW FEATURE
This part focuses on the workflow to implement a new feature. From branching away from master until it is ready for the Pull Request.
GIT BRANCHING MODEL
At Codeship we follow the GitHub-flow model of development (click to check out Scott Chaconâs article), so whenever we start a feature we create a feature or bug branch. Most of our team uses git-extras by visionmedia for this.
Typically only one person works on one branch. If we need more people to work on a feature we break it down to the smallest possible chunk that one person can ship.
Take this example: Ben implemented a first basic version that allowed to invite collaborators from Github to Codeship. He worked on his own feature branch and had a simple UI that was ready to be shipped. After the feature passed the Pull Request and Code Review his changes were shipped. Later on Alex created another feature branch from master
and implemented the final user interface which makes it very easy to invite anyone to the Codeship who committed to the GitHub repository.
Both, Ben and Alex, committed and pushed regularly while still working on the feature. When a small piece is done we typically push it to Github to run our complete test suite.Â While Codeship runs all of the integration tests we keep working on the feature.
This way we very quickly see if our changes broke any part of the application without running the full test suite locally. And breaking a feature branch is absolutely ok. We want our developers to push early and often and let the Testing Server take care of the tests so they do not waste time.
There are numerous advantages in shipping a minimum viable feature first. We keep waiting times between the developers to a minimum while still shipping improvements very quickly. Thereby we remove a lot of unnecessary communication. And we never run into any kind of merge problems when two developers work on the same feature.
Of course there are challenges with this Workflow. Sometimes features are shipped with the expectation that they are improved right afterwards, but something else needs immediate attention. This way the improvement can take a bit until it is shipped.
Therefore getting that minimum viable feature right is very important. Big enough to be valuable, but small enough toÂ be shipped fast and by a single person.
PULL REQUESTS AND CODE REVIEW
In the previous part we explained how we start with a new feature and the workflow to implement it. This part focuses on the steps necessary to get your feature ready to be shipped.
USING PULL REQUESTS
As mentioned we use GitHub Flow on GitHub. But the whole workflow we talk about in this ebook is also possible when working with Bitbucket.
We do not have a policy when a Pull Request should be opened. Some of our developers open them when they start a feature, some wait until the feature is implemented. We then push regularly to that branch as explained a few pages earlier.
Open Pull Requests are helpful as everyone can see what the other members of the team are working on. One really important part of feature branches and Pull Requests are proper commit messages.
WRITING GOOD COMMIT MESSAGE
As a distributed version control system, git makes it easy to take large changes and split them into small commits. These commits have a descriptive message with a short explanation in the first line and, if necessary, a longer description after a separation line. The git documentation project has some nice guidelines on it. Giving every commit a proper message and splitting large changes into a number of small commits makes your code history easier to read. It will be easier for your teammates to understand the history of your code.
We prefer git-gui for committing and splitting up commits. It does only this job, but it does it very well. Of course there are some other git tools. Find out which you like best!
USING GITHUB ISSUES AND LABELS
Once the feature is ready to be reviewed we assign its Pull Request to somebody else on the team. You have to read through the changes on GitHub at least once before handing the Pull Request. Making sure no obvious errors are in the code reduces the time and cycles necessary for a good code review.
It is very helpful to inspect the changes outside of your own editor and with the excellent GitHub comparison view.
Now that the Pull Request is assigned, somebody on our team has a GitHub issue with his name attached. This way anybody can easily filter out the Pull Requests that needs to be looked at. The last step is labeling the Pull Request with needs review. This way the reviewer knows this Pull Request is ready to be reviewed.
When starting a Pull Request, the developer and reviewer can decide on looking through the Pull Request together or not. Especially for large changes reviewing the code as a pair is great.
We review the structure and syntax of the code, but we currently do not review the functionality itself. Before implementing a feature we discuss it in detail, so there is a common understanding. Once the feature is implemented, we trust our engineers that it does what it should. We usually donât do feature reviews on our staging system.
Of course you can do functional reviews for large and public facing features. Just so a second pair of eyes looks at changes that possibly impact a lot of users. When the review is done it either gets merged or, if the reviewer leaves comments, the GitHub issue is labeled as reviewed. This way developers can take a look at their open issues and see which have been reviewed and which need more attention.
The feature is ready to be merged as soon as the developer either updates the Pull Request or explains why she did it differently.
Thereâs one challenge with every release workflow that involves code reviews: Getting the team to review as fast as possible without interrupting their work. We try to do it asynchronously for smaller features and sitting down for shared code review on larger features. This is no silver bullet, but in our experience it works well.
“It is very helpful to inspect changes outside of your own editor.”
GITHUB MERGE BUTTON
To make sure nobody introduces changes into our production system without review, we have a strict rule that you can never merge your own code. This rule is not enforced through technology, but a strict team policy.
The only exceptions are scheduled releases, for example a maintenance deployment. If this happens we let somebody review the changes and leave a ready to be merged comment. The build can then be merged whenever the developer wants to.
By releasing through the GitHub merge button everyone on the team can push to production. Making this possible for every developer or designer in your team is very valuable. It allows everyone to take part in the process and it speeds up the daily work.
As soon as we hit the merge button our tests are kicked off on our Testing Server and the deployment starts.
DEPLOYMENT PIPELINES & ZERO DOWNTIME DEPLOYMENTS
In the previous part we talked about how we review code, open Pull Requests and use Github issues to manage our Development Workflow. This part will show every step that happens after a Pull Request is merged into our master branch. We use an automated deployment pipeline for releasing our code into production.
A Deployment Pipeline lays out the whole process that your code needs to go through from your repository to production.Â It breaks the build into several parts (e.g. build, test and deploy) and all the associated steps that need to be taken. By defining a pipeline it is always clear which step needs to happen next. Martin Fowler describes it really well in his blogpost.
If you want to dig deeper into Deployment Pipelines we recommend Jez Humble and David Farleyâs book: Continuous Delivery.
CONFIGURE DEPLOYMENTS PER BRANCH
To automate deployment to different environments we have found that it works best to define actions for different gitÂ branches. If you always push the latest commit of your production branch to your production environment, it is very easy to determine what is currently deployed by just looking at the git branch. Git and other source code management systems only permit one commit at the top of a branch, so there can be no confusion.
At Codeship we deploy our master branch automatically to production. Many of our customers deploy the master branch to a staging environment and a production branch to their production environment. A simple git merge andÂ git push, or a Github Pull Request, is their way of releasing their changes.
One problem with this approach is that branch names have to be meaningful. Having a development branch which is deployed to staging and a master branch that gets deployed to production can confuse new team members. Naming branches that get deployed âproductionâ or âstagingâ is more intuitive.
âMasterâ is a convention in git and should be kept, but dedicated branch names are easier to understand in a Deployment Pipeline.
As soon as the feature branch is merged into our master, a new build is started on our Testing Server. We run the same test commands as we did on the feature branch to make sure there are no problems in the merged version.
When all tests pass for the master branch the deployment starts. Before pushing to production we want to make sure that all database migrations work and that our app starts successfully.
First we deploy to staging. Then we run our current set of migrations. We copy our production database to staging onceÂ a day (click to read a blog post about how we do this). Therefore, when we run migrations on staging, the database is very close to our production database. This allows us to make sure our migrations work, before deploying to production.
The last step in our staging deployment is calling the URL of our staging site to make sure it started successfully. Wget and its retry capabilities make sure the website is up and running.
Afterwards we run our migrations in a separate step after the deployment and clear our cache.
We then repeat the whole process for our production system. We push to the Heroku production repository and check that the site still works. An enhancement would be to have tests that run against the deployed version, but so far we havenât had any problems without these tests. Our extensive Cucumber/Capybara test suite has caught all problems so far.
There is one slight difference between our staging and production deployment though:
Zero Downtime Deployments
As we want to deploy several times a day without any downtime we use Herokuâs preboot feature. We started using it at the beginning of this year. Whenever we push a new release, it starts this release on a second server and switches the routing to it after about 3 minutes.
The downside is that Zero Downtime Deployments require more care with database changes. As two versions of your codebase need to be able to work at the same time you canât just remove or rename fields. Renaming or deleting a column or table needs to be spread out over several deployments. This way we make sure that the application still works with every incremental change.
It is important to automate every step of the deployment. No matter if you want to deploy your code on every merge to master or trigger it manually by merging the master into another branch.
Now that youâve seen our workflow in detail, the main take away we want you to leave with is that process and thinking about process is the lifeblood of every team. Technology comes and goes, but only through a great process can you build something amazing. Taking these steps to eventually come to Continuous Deployment will makeÂ you and your team more productive, successful and in the end also happy.
Â ABOUT THE AUTHOR
This article is written byÂ Florian Motlik is CTO of Codeship.Â During his Software Engineering studies and work at theÂ University of Technology Vienna Flo came into contact withÂ proper Software engineering, quality and process.