Shifting UI tests to the far left (part 2)
How you write specifications using UI tests before the development even starts
In the first part of the article, we established not only that UI tests can be used to write specifications for mobile apps, but that they can also be very expressive in doing so, allowing for accurate modelling of the behaviour of the system and leaving no room for overlooked specifications.
If you were though to use UI tests to write specifications, then you would also need to have a way to do so before the development even starts while still in the specifications phase. Let’s see how we can do this.
Before embracing the “UI tests as specifications” approach, the timeline of a feature release in our team, from the requirements phase to the actual release looked like that.
The product team outlined requirements, which business analysts refined through discussions with developers and stakeholders. They clarified designs, created documentation, and incorporated BDD scenarios. Development prioritised unit and snapshot testing, with UI tests addressed at the end, based on team availability.
UI tests were essentially a 2nd class citizen due to time constraints and maintenance challenges. With the goal of turning them to a 1st class citizen we shifted them to the far left, becoming part of the specifications phase and replacing the BDD scenarios.
Writing UI tests becomes now the responsibility of the feature’s tech lead, who should have the necessary Android and iOS skills to write them. This shift eliminates reliance on developer availability, as the UI tests are now written before the development begins. Specifications now become acceptance criteria and regression tests at the same time, making it easier to sync the implementations between Android and iOS.
Open the specs MR
Before any development starts — remember we are still in the specifications phase trying to write the specifications for our engineers — the feature’s tech lead opens the very 1st MR writing the specs of the feature as UI tests in either iOS or Android.
Apply code ownership
The specs MR contains only the the UI tests files themselves, all placed within the a folder that is under the code ownership of the feature’s tech lead.
And this is quite important because if ever the behaviour of the app was to deviate from the initial specs due to a code change, then the relevant UI tests would fail and we would have to update them. And these updates would need to be approved by the code owner, effectively the owner of the specs themselves, making him aware of any change in the specifications and mismatches between specification and implementation for free.
And with source control you even have a history of these changes to reason about if needed. The need to communicate in order to be in sync about any changes in specs vanishes into thin air 💨.
And then the feature’s tech lead starts writing the actual scenarios.
Define the preconditions (network stubs and feature flags)
We should start each scenario by leaving a note to the developer what should its preconditions be (the network stubs1 of the API calls that we care about and any feature flag setup that you might have). In the scenario we described in part 1 of the article, we specifically care about the two login API calls and we require them to return successfully. All other calls are irrelevant (and are stubbed successfully by default by the way).
When writing specs you do not have to implement the stubs — the developer will. You just have to make it explicit for him in the comments section like above.
Likewise, in non-happy path scenarios, you should make it clear in the comments which API call fails and the relevant error code.
Write the scenario using the Page Object pattern
Then you can begin writing the scenario itself using the Page Object pattern which, as we have seen, is very close to the clarity of the natural language used in a BDD scenario.
Use empty stub methods
The methods that we define for each screen (like the LoginScreen in the above example) should be empty stub methods. And by empty I mean that you do not care if the tapLoginButton method is actually implemented. The developer will have to implement it when he starts working on this feature, trying to make the test pass. In a TDD fashion but with UI tests if I may say2.
Do not leave your API calls unspecified (optional)
If you wanted to specify which API calls should be part of this scenario and in what order, you can add them by defining how the snapshot of the API call interaction should look like as we saw in part 1 of the article.
Address any question raised during MR code review
Writing specs that way we use a language the devs completely understand. In the tool they are most familiar with. When all scenarios are done, you can proceed with requesting feedback on the scenarios and the specs, just like when requesting feedback for a code review.
When all the discussions are resolved, the specs MR is merged and the development can start. Eventually, the developers will have to make all the tests pass, thus making sure specs are respected.
Copy the UI tests for Android as well
And in the native mobile development world where the duality between Android and iOS is omnipresent , syncing the Android implementation becomes as easy as copying the UI tests for Android and applying the same code ownership rules 🎉🎉.
It should be clear by now that writing specifications using UI tests before coding greatly reduces any communication flaws in the specifications phase, the first most important problems when writing specifications. The main benefits include:
Leaves no ambiguity to the developers
Specs that never go out of sync with the implementation
Under source control and code ownership, we can keep track when a requirement is changed w/o any communication needed
No more “Oh I have to update the BDD scenarios” or “Is this specification scenario written here still valid? Who should I ask?” or “Does it behave the same in Android and iOS” moments. And then hope that actual implementation in both Android and iOS also follows the specification in question.
As a project lead, opt for specifications as acceptance criteria and regression tests at the same time, opt for peace of mind 🧘
Using network stubs and not the actual network gives us control of the responses and also eliminates the delay and flakiness introduced by the real network.
Great article George, I want to emphasize my question on "Leaves no ambiguity to the developers".
A function name and parameters in the Page Object Pattern may not be enough for the developer to understand what it is requested. In that case do you write comments next to the method or you use a different way?
On top of that, to make it more practical, for example assume you have a screen with 3 labels and we want to assert them. According on PageOP you would write something like:
robot
.assertLabel1(is: "1")
.assertLabel2(is: "2")
.assertLabel3(is: "3")
Now the developer clearly understands what happens but the time performance of the UITest is not good. It's not good because we cannot let the system check the predicates in parallel. Instead we check them sequentially (Check this (speed section) https://medium.com/trendyol-tech/how-to-make-ios-ui-testing-fast-and-reliable-6f572a0955f2 )
So how do you tackle this problem. Would you sacrifice readability over performance, or the other way around?