We have a couple of large, complicated applications with large, complicated user interfaces. We’re set up with Codeship for continuous integration (Codeship is pretty great, btw). Our unit tests run on every push, and they’re nice and fast - we’re using a stub sqlite database for testing, and using a few other tricks for speeding up tests that I’ll talk about in another post. We also have a rather large suite of UI tests built in Behat. These tests are much, much slower to run and slow down the continuous integration considerably. We also don’t necessarily need them to run on every push to dev.
Turns out Codeship makes the commit message available in an environmental variable called CI_MESSAGE. I wrote a very simple bash script that looks like this:
ci_message=`printenv CI_MESSAGE`; if [[ $ci_message == *"--ui-test"* ]] then npm install -g http-server protractor webdriver-manager update nohup bash -c "webdriver-manager start 2>&1 &" && sleep 9 bin/behat --profile=codeship --stop-on-failure; fi
This script will parse the commit message, and if it sees the ‘—ui-test’ flag it’ll install the necessary tools and run the UI tests. We just call that script in our test commands in Codeship’s setup. We’re in the processes of setting up with Saucelabs so that we can run the tests there. That will speed up the tests a bit since we won’t have to install anything to run them, and has the huge bonus of providing a video of each test so we can see where/when it fails in the case where a test passes locally but fails in the CI environment. It’s fairly primitive at the moment, but I have plans to fancy it up and allow for things like passing in a particular Behat profile or test suite to run, and perhaps setting it so that the full suite of tests only runs when merging dev up into the staging branch.