Andrew Cavanagh

Developer.  Project Architect.  Huge Nerd.

(not necessarily in that order)

Behat is a great system for acceptance testing. Many of our tests require a browser with a fully functional javascript engine. There’s a number of ways to get this to work on CI systems, but most of them are running the test in sort of a black box. I find that when a behat tests fails it’s sometimes very very helpful to be able to see what was actually happening in the browser when it failed. Enter Saucelabs. If you’re running the tests via Saucelabs you get a nice video of the whole test running in the browser, so you can review and see what happened when it failed. You will need a saucelabs account, but they’re at the time of this writing you could get a free account for low tier use. You also need to download sauceconnect.

Because our suite of UI tests takes a really long time to run we don’t need it to happen on every push. To that end, we’ve scripted it out to only run on commits with a flag in the commit message - see this post for details. We have a copy sauceconnect in a folder with all of our deployment scripts, which is under version control - you could also include a step to download and unpack it in your CI setup or scripts if you prefer not to have it hanging around in version control. The steps look like this:

  npm install -g bower //we need this to install bower, obviously

  bower install // we need bower to install all of the JS libraries the project relies on

  chmod +x deployment/sc-4.3.11-linux/bin/sc //this makes the sacueconnect script mentioned above executable

  nohup bash -c "deployment/sc-4.3.11-linux/bin/sc -u $MY_USERNAME -k $MY_KEY &" && sleep 20; //this runs sauceconnect.  You can get your usrname and key from your saucelabs account.

  bin/behat --profile=saucelabs --stop-on-failure --tags="~@wip";  //this runs the behat tests with the saucelabs profile, excluding tests with the @wip tag.

The saucelabs profile looks like this:

saucelabs:
    extensions:
        Behat\MinkExtension:
            base_url: http://localhost:8080
            selenium2:
              wd_host: '$MY_USERNAME:$MY_KEY@ondemand.saucelabs.com/wd/hub'
              capabilities:
                platform: 'Windows 8'
                browser: chrome
                version: '33'
                name: '$MY_APP'

comments powered by Disqus