Acceptance testing iOS apps

I recently had to develop a update for an iOS app. I’ve become accustomed to the excellent continuous build & integration environment we make use in our workplace for our (primarily Ruby-based) projects. This led me to investigate tools that I could use to develop an update to the iOS app in a similar way.

One area I was particularly interested in was acceptance testing the app. We have fairly extensive API tests in place to ensure core user-flows such as signup and artist tracking should work with the app, so I was interested in how we could acceptance test our iOS app.

After experimenting with a bunch of existing tools, including Frank and Zucchini (both great tools, but with limitations on running tests on hardware devices, and a little fiddly to setup), I settled on using Apple’s own Instruments application to record tests, and the excellent Tuneup_js JavaScript library to help format the recorded tests into separate test files.

What I needed was a way of running the recorded tests in an easily repeatable fashion, from the command-line (to allow me to wire the acceptance tests up to a Jenkins task), and to handle the other steps outside of the scope of Instruments and Tuneup_js, namely:

  • Clean any previous app installs from the target test device
  • Build the latest version of the app
  • Install the app on the target test device
  • Run Tuneup_js-formatted tests against the installed app
  • Report the test results

I wrote a simple Ruby application, Scenarios, to handle this process for me. Scenarios can be run from the command-line (against the iOS simulator or real hardware), and includes support for auto-detecting connected hardware, as well as adding a couple of extra functions on top of the UIAutomation library:

  • withTimout: use this for potentially long-running calls (e.g. a button click to login, where the API may be slow)
  • slowTap: Allows for slowing-down the speed at which taps are executed. This helps to a) see what tests are doing, b) help mimic human behaviour (those tests can move pretty fast).

Here’s a video of the sample running on a simulator:

If you get a chance to try it out, let me know how you get on :)