Embedded web server for iOS UI testing
If you’ve ever visited some of the tech companies around Silicon Valley, you may have seen an iPad stand at the front desk with something like this on the screen:
Look familiar? That’s Envoy, our visitor registration app, which has signed in more than 5 million visitors in three years at companies all around the world.
Recently, we sometimes found ourselves trapped in a loop of fixing bugs instead of creating new features for the iPad app. The fundamental problem here is that we’ve incurred too much technical debt.
Software development is hard. To deliver the highest quality software in the industry and continually evolve, we need to pay back the technical debt we incurred by adopting automatic tests.
Mocking is hard
To write unit tests for the legacy code base, we need to refactor the design objects and architecture to make them testable first. However, without existing tests to ensure all functionalities are still working as expected, it’s difficult to refactor. As it turns out, it’s a chicken or egg problem.
Fortunately, in addition to writing unit tests against classes, we can write tests for UI interaction against the whole app. As long as we have good coverage from the UI surface down to the app, we can then refactor the legacy code inside the app later without worrying too much about breaking it.
Although UI testing solves the chicken or egg problem of testing and refactoring, it’s not an easy thing to do. The major issue we’ve encountered is that due to the way UI testing works, there are two standalone processes: 1) the UI testing runner and 2) the target app. The UI testing runner launches the target app and uses the accessibility infrastructure to inspect and control the app.
To test our app, we need to control the data returned from the API client object, but as they are two standalone processes, you cannot control in-memory values directly from one process to another. You’re unable to do this:
Then replace the API and set the mock data to return:
Iteration #1: Pass environment variables
Despite that you cannot mock the API directly (like what you do for unit tests), you can pass environment variables and arguments to the target app when you launch it like this:
With that in mind, our first approach was to write a mock API client that mocks API calls according to environment variables.
Passing environment variables and mocking the API function is easy and works, but there are many drawbacks to this approach:
- Need to write code for reading environment variables and mocking the API
- Hard to control the API, like delaying the response or no response to test timeout handing
- API behavior is not customizable. For example, if we want to return different values according to the received parameters
- Hard to check API calls and the data passed in
Iteration #2: Embassy and Ambassador
After writing a few test cases with the environment variables approach, we realized that it’s obviously not the best way to write tests. In the end, what we mocked is an APIClient that connects to the HTTP server, but then we thought: why not just mock the HTTP API server instead?
We looked around to find open source HTTP servers written in Swift, but since Swift is a relatively new programming language and Apple just open sourced it a while ago, the resources for server-side Swift were very limited. We did find swifter, however, it’s a sync style server and we want async. We also found Perfect and Kitura, but they‘re both geared towards being the production ready web application solution in Swift, which is too much heavy weaponry for us.
You may ask why not use existing resource from other language community, certainly we can, but as if it’s not in Swift, it brings extra burden for our iOS engineers to context switch between different languages. And there are things you cannot do or hard to do, like in-line assertion in the API response handler just right inside the test case.
Since we couldn’t find anything that met our needs, we finally decided to build the wheel ourselves. It’s called Embassy, a super lightweight async HTTP server built purely in Swift — and of course, it’s open source.
Some awesome features:
- Super lightweight, only 1.5 K lines
- Zero third-party dependency
- Async event loop-based HTTP server, makes long-polling, delay and bandwidth throttling all possible
- HTTP Application based on SWSGI, super flexible
- IPV6 ready, also supports IPV4 (dual stack)
For this HTTP server, we defined a gateway interface called SWSGI, a hat tip to Python’s WSGI (Web Server Gateway Interface). It’s basically a simple function defined as:
It decouples web applications from the implementation details of the web server and also allows middleware to be wrapped around other web applications. It’s very easy to set up an HTTP server and run it inside your UI tests. Dealing with SWSGI is also easy, but to make mocking APIs even easier, we built Ambassador, a lightweight web framework based on SWSGI.
Automatic UI testing in action
With Embassy and Ambassador, mocking APIs and writing UI tests couldn’t be easier. You can install them with CocoaPods: add Embassy and Ambassador to your Podfile target to the UI test like this:
and run “pod install.”
Here’s an example how you run a simple HTTP API server:
Next, configure your app to connect to “http://localhost:8080/api/v2/users” instead of the real API server. It should be able to get the provisioned JSON payload, which looks like this:
To avoid writing the handlers for the same endpoints again and again, we provide a DefaultRouter that has decent default handler build-in, like so:
Then, we have a base class UITestBase for all UI tests cases to inherit:
As you can see, we pass ENVOY_BASEURL here as the API base URL for your own API client. You also need to make it read from the environment variable.
Finally, you can now write test cases with mocked APIs:
You simply overwrite the endpoint, you can assert and whatever you want in the response handler.
More than just mocking APIs
One good thing about bringing embedded web servers inside UI tests is that you can also mock other services, like PubNub, we use it to allow server to notify the iPad app for certain events. We use a classic HTTP long polling technique introduced in the Ajax/Comet era to allow the test runner to push messages to the target app in real-time. This way, the test cases can push a message to the app whenever they want.
As the iPad app supports badge printing, we also created a protocol for badge printer and created a mock client connecting to an HTTP API endpoint for printer instead of real printer. This means we can examine the printed document from the iPad.
Wanna build cool stuff with us? Envoy is hiring
As a language, Swift was introduced mainly for building iOS apps and desktop apps. Currently, there are limited resources available, but we see huge potential in bringing server side technology into the Swift community.
The SWSGI gateway interface we defined for Embassy demonstrates how the server side ecosystem can be built on top of it. What’s great is that this was originally a 20% time project and meant for UI testing purposes only.
These are just a few of the cool things we’ve built at Envoy. If you also enjoy building cool stuff and solving hard problems, Envoy is hiring. Here are the open engineering positions:
Feel free to shoot us your resume if you’re interested. Thanks! :)
