Jekyll2022-02-28T22:01:15-05:00https://alexilyenko.github.io/feed.xmlAlex Ilyenko’s BlogSDET's BlogAlex IlyenkoDesign Patterns in Test Automation III2018-02-28T00:00:00-05:002018-02-28T00:00:00-05:00https://alexilyenko.github.io/patterns-3<p>I already wrote about using <a href="https://alexilyenko.github.io/patterns-1/">Structural</a> and <a href="https://alexilyenko.github.io/patterns-2/">Data Patterns</a> in your automated tests before. That’s why if you’d like to learn more, you should check those posts out. In today’s last part of the series we’ll be talking about Technical and Business Involvement patterns.</p>
<h2 id="technical-patterns">Technical Patterns</h2>
<p>The main goal of Technical Patterns is to encapsulate technical details from test logic, providing extra low-level control over them.</p>
<h2 id="decorator">Decorator</h2>
<p>Decorator is a very well-known pattern, since it was mentioned in the GoF list and discussed in many other programming books and articles. The example of this one is simple. Let’s imagine you’re working with any driver implementation (e.g. WebDriver) and are willing to add an extra functionality to it, like logging or caching. But in the same time you don’t wanna reveal that additional functionality to your actual tests, leaving test logic the same as it was before. That’s where you want to use Decorator.</p>
<p>Decorator helps to implement so called “Cabbage principle”, when you are able to wrap one driver implementation into another, in the way cabbage leaves are formed. Your tests won’t be aware of that extra layer since they work with the same interface as before.</p>
<p>For instance, you want to log every click on some element in your tests. All you need to do is to decorate your initial <code class="language-plaintext highlighter-rouge">WebDriver</code> object, by wrapping it into the <code class="language-plaintext highlighter-rouge">EventFiringWebDriver</code> and registering new listener, while your tests don’t have to be changed at all:</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">new</span> <span class="nc">EventFiringWebDriver</span><span class="o">(</span><span class="n">driver</span><span class="o">)</span>
<span class="o">.</span><span class="na">register</span><span class="o">(</span><span class="k">new</span> <span class="nc">AbstractWebDriverEventListener</span><span class="o">()</span> <span class="o">{</span>
<span class="nd">@Override</span>
<span class="kd">public</span> <span class="kt">void</span> <span class="nf">afterClickOn</span><span class="o">(</span><span class="nc">WebElement</span> <span class="n">element</span><span class="o">,</span>
<span class="nc">WebDriver</span> <span class="n">driver</span><span class="o">)</span> <span class="o">{</span>
<span class="no">LOG</span><span class="o">.</span><span class="na">log</span><span class="o">(</span><span class="nc">Level</span><span class="o">.</span><span class="na">INFO</span><span class="o">,</span> <span class="s">"Click on element "</span>
<span class="o">+</span> <span class="n">element</span><span class="o">.</span><span class="na">getTagName</span><span class="o">());</span>
<span class="o">}</span>
<span class="o">});</span>
</code></pre></div></div>
<p>To get rid of this kind of wrapping in the test logic completely, one could leverage something like <a href="https://en.wikipedia.org/wiki/Factory_method_pattern">Factory Pattern</a>, when user can get the browser/driver simply requesting it from the Factory. Also using the <a href="https://alexilyenko.github.io/patterns-2/#object-pool--flyweight">Browser Pool</a> might be not a bad idea too.</p>
<h2 id="proxy">Proxy</h2>
<p>Proxy is the pattern that allows to intervene into the process running between you and another user, introducing new logic in-between without affecting either of the sides.</p>
<figure class="">
<img src="/assets/images/proxy.png" alt="" /></figure>
<p>This pattern might be useful when you, for example, want to add logging, to enable or to disable something, to have control over some additional recourses etc. The most popular method of using it in tests is to set up the HTTP proxy. It allows to dynamically enable and disable host blacklists, excluding or stubbing the third-party sites like Facebook or Twitter in your tests. Sometimes it’s the only way you can check some exceptional scenarios for the external services like those ones.</p>
<p>Another two examples I could come up with are caching of nonfunctional assets (like images or CSS), which do not affect the functionality of the app, and collecting http-traffic for further analysis while running the tests (by analysis I mean verifying if some images were missing or if any recourses took a lot of time to load etc).</p>
<p>Use HTTP Proxy for tests to:</p>
<ul>
<li>Blacklist external resources (Facebook, Twitter, Ads etc)</li>
<li>Cache images and other nonfunctional assets</li>
<li>Collect HTTP traffic for analysis (loading time, 404 errors, redirects etc)</li>
<li>Speed up page loading</li>
</ul>
<h2 id="business-involvement-patterns">Business Involvement Patterns</h2>
<p>The last group of patterns I was going to review is Business Involvement Patterns. The main goal of these patterns is to bring Product Owners, Business Analytics and other people, responsible for requirements, as close to test automation as possible. This way its benefits might become crystal clear to them and they would get involved into it as much as we are. It’s the perfect scenario for all the members of the team, isn’t it?</p>
<h2 id="keyword-driven-testing">Keyword Driven Testing</h2>
<p>To make the latter possible we could use the popular pattern, which allows to distance tests from the code and helps to make it possible to write them in human-readable language, understood to regular person.</p>
<p>Keyword Driven Tests are written using keywords - domain commands, clear to everyone in the team. Hence they might contain some data, they should be implemented by people, who’s familiar with the technical details and are able to design them in code. But in the same time the framework should allow anyone to write the tests - it might be manager, business analytic or test engineer without any coding experience.</p>
<div class="language-text highlighter-rouge"><div class="highlight"><pre class="highlight"><code>*** Settings ***
Documentation A test suite with a single test for valid
login.
This test has a workflow that is created
using keywords in the imported resource
file.
Resource resource.txt
*** Test Cases ***
Valid login
[Set Up] Open Browser
Open Login Page
Input Username someUser
Input Password somePassword
Submit Credentials
Welcome Page should be Opened
[Teardown] Close Browser
</code></pre></div></div>
<p>In the example above, you can see how this pattern is implemented with the help of <a href="http://robotframework.org/">Robot Framework</a>. There are lots of free and paid frameworks out there, which help to adopt KDT approach and make Business’s life easier.</p>
<p>The main idea behind this is that there is finite number of operations in any app. This means that if all of them are implemented as keywords, we are able to build infinite number of test scenarios by combining them in one or another way. Doing this you could provide the working tool to someone who wants to be involved into the building of the automation on the project.</p>
<p>This pattern faced a lot of critics from people who implemented it in their automation but weren’t able to involve anyone from business side into writing the tests. That’s why before implementing Key Driven Testing on your project you should discuss its usage with all the stakeholders. Otherwise it might be overhead to build something complex like this without business awareness and support.</p>
<h2 id="behavior-specification">Behavior Specification</h2>
<p>The same is true for the next pattern we’re going to discuss - Behavior Specification. It suggests to transform the writing of the tests into defining of the expected behavior. This way the feature can be described in the form of behavior scenarios. Here is the simple example of this pattern:</p>
<div class="language-gherkin highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">Feature</span><span class="p">:</span> Addition
In order to avoid silly mistakes
As a math idiot
I want to be told the sum of two numbers
<span class="kn">Scenario</span><span class="p">:</span> Add two numbers
<span class="nf">Given </span>I entered 50 into the calculator
<span class="nf">And </span> I entered 70 into the calculator
<span class="nf">When </span> I pressed Add button
<span class="nf">Then </span> The result should be 12 on the screen
</code></pre></div></div>
<p>The scenario describes simple workflow of addition of two numbers using calculator buttons and verification of the result on its screen. This test is not a “test” in a common sense, it’s more a behavior specification. It’s easy to read and can be understood by anyone. But again, this pattern will be super useful only if the business wants to be involved in the testing process (e.g. by creating the acceptance criteria based on those specs and tracking the results of tests).</p>
<p>If the business on your project is not ready for this kind of approach, you don’t have a problem which needs the given solution, and by implementing it you’d create another unnecessary layer of abstraction above your test logic.</p>
<h2 id="behavior-driven-development">Behavior Driven Development</h2>
<p>Usually Behavior Specification is used along with concept of Behavior Driven Development, which is saying that:</p>
<ul>
<li>at first we need to describe the behavior of functionality as a spec</li>
<li>next we want to write a test for that</li>
<li>then we need to implement that functionality</li>
<li>verify that test passes</li>
<li>verify that behavior scenario passes</li>
</ul>
<figure class="">
<img src="/assets/images/bdd.png" alt="" /></figure>
<p>As a result we’ll get working low-level tests and comprehensive high-level scenarios, making developers and business happy, right? But I’m afraid its not that simple. It would be complete waste of time if this approach is used by engineers (technical people) exclusively without involving any business to writing scenarios.</p>
<h2 id="steps">Steps</h2>
<p>The last for today pattern’s idea might be pretty interesting regardless what approach you’re using: Behavior Driven, Keyword Driven Development or Behavior Specification. When we’re talking about logic scenarios, we think about them as if they consist of steps. But when you implement them in your code, these steps often get lost among all the technical details, data manipulations and other things we’re doing in our tests. It would be great if we could isolate those steps from other code showing what this particular test does and easing the hurdle of its maintenance. Steps pattern will help us to do that!</p>
<p>Your test steps might be divided into groups by functionality and gathered into the special classes or methods. If you don’t have the step you need in your test, you can simply create one. You can use the Steps Pattern either with some technical framework (e.g. <a href="https://alexilyenko.github.io/patterns-1/#page-object">Page Object</a>) or without any. It doesn’t matter as long as you divide your logic into the Steps:</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">public</span> <span class="kt">void</span> <span class="nf">bookShouldBeSearchedByName</span><span class="o">()</span> <span class="o">{</span>
<span class="n">loginToBookStore</span><span class="o">(</span><span class="s">"u"</span><span class="o">,</span> <span class="s">"pass"</span><span class="o">);</span>
<span class="n">openBookSearchPage</span><span class="o">();</span>
<span class="n">enterSearchCriteria</span><span class="o">(</span><span class="s">"name"</span><span class="o">,</span> <span class="s">"selenium"</span><span class="o">);</span>
<span class="n">assertFalse</span><span class="o">(</span><span class="n">isEmptySearchResult</span><span class="o">());</span>
<span class="n">openBookDetailsByIndex</span><span class="o">(</span><span class="mi">1</span><span class="o">);</span>
<span class="n">assertEquals</span><span class="o">(</span><span class="s">"Selenium 1.0 Testing Tools"</span><span class="o">,</span> <span class="n">getBookTitle</span><span class="o">());</span>
<span class="n">assertEquals</span><span class="o">(</span><span class="s">"David Burns"</span><span class="o">,</span> <span class="n">getBookAuthor</span><span class="o">());</span>
<span class="o">}</span>
</code></pre></div></div>
<p>The Steps example above can be easily transformed to a Test Scenario:</p>
<div class="language-text highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Steps:
1. Login to book store as user 'u' and password 'pass'
2. Open search books pages
3. Try to search by word 'selenium'
4. Results should not be empty
5. Open the first found book details
Expected results:
1. Book title is 'Selenium 1.0 Testing Tools'
2. Author is 'David Burns'
</code></pre></div></div>
<p>Writing test scenarios in this way may help to avoid using of any extra Test Case Management system, since all of the Test Steps, either automated or manual ones, are stored in one place, which is your framework. As a bonus now you can easily track, how many steps are automated, how many of them are broken, who was responsible for their creation etc.</p>
<p>It’s pretty simple to write a great test. The other thing is to write dozens of them while the functionality of your app is scaling, the number of people on your project responsible for the tests is growing, without losing the speed of development and avoiding the time wasting on supporting of those tests etc. The latter can be done leveraging the Design Patterns discussed in the <a href="https://alexilyenko.github.io/tags/#patterns">Patterns</a> series in this blog. If you know how and when to use them, your tests will be faster, more effective, more scalable and more reliable.</p>
<ul>
<li><a href="https://alexilyenko.github.io/patterns-1/">Design Patterns in Test Automation I</a></li>
<li><a href="https://alexilyenko.github.io/patterns-2/">Design Patterns in Test Automation II</a></li>
</ul>
<p><a href="https://alexilyenko.github.io/"><img src="https://alexilyenko.github.io/assets/images/share_message.png" alt="Feel free to share!" /></a></p>Alex IlyenkoTechnical and Business Involvement PatternsDesign Patterns in Test Automation II2018-01-03T00:00:00-05:002018-01-03T00:00:00-05:00https://alexilyenko.github.io/patterns-2<p>In the previous post we discussed <a href="https://alexilyenko.github.io/patterns-1/">Structural Patterns in Test Automation</a>, which are very valuable for building robust and scalable test frameworks and tools. In this post we’ll talk about Data Patterns.</p>
<h2 id="data-patterns">Data Patterns</h2>
<p>The main goal of data patterns is to split data and test logic as well as to reduce boilerplate and code duplication in our tests. It should make them more understandable and easier in maintenance for anyone who works with them.</p>
<h2 id="value-object">Value Object</h2>
<p>We will start from the easiest pattern in this group - Value Object. I think most of us used it once in a while, but unfortunately I’ve seen lots of the projects, where this best practice is neglected by developers. That’s really sad because from design perspective Value Object can make your code more readable and significantly reduce amount of repeatable constructions.</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">public</span> <span class="kt">void</span> <span class="nf">createUser</span><span class="o">(</span><span class="nc">String</span> <span class="n">firstName</span><span class="o">,</span> <span class="nc">String</span> <span class="n">lastName</span><span class="o">,</span>
<span class="kt">int</span> <span class="n">age</span><span class="o">,</span> <span class="kt">boolean</span> <span class="n">isMarried</span><span class="o">,</span> <span class="nc">List</span><span class="o"><</span><span class="nc">String</span><span class="o">></span> <span class="n">accomplishments</span><span class="o">)</span> <span class="o">{</span>
<span class="n">enter</span><span class="o">(</span><span class="n">firstName</span><span class="o">,</span> <span class="n">into</span><span class="o">(</span><span class="s">"name"</span><span class="o">));</span>
<span class="n">enter</span><span class="o">(</span><span class="n">lastName</span><span class="o">,</span> <span class="n">into</span><span class="o">(</span><span class="s">"lastName"</span><span class="o">));</span>
<span class="n">enter</span><span class="o">(</span><span class="n">age</span><span class="o">,</span> <span class="n">into</span><span class="o">(</span><span class="s">"age"</span><span class="o">));</span>
<span class="n">enterMaritalStatus</span><span class="o">(</span><span class="n">isMarried</span><span class="o">);</span>
<span class="n">accomplishments</span><span class="o">.</span><span class="na">forEach</span><span class="o">(</span><span class="k">this</span><span class="o">::</span><span class="n">addAccomplishment</span><span class="o">);</span>
<span class="o">}</span>
</code></pre></div></div>
<figure class="">
<img src="/assets/images/arrow.png" alt="" /></figure>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">public</span> <span class="kt">void</span> <span class="nf">createUser</span><span class="o">(</span><span class="nc">User</span> <span class="n">user</span><span class="o">)</span> <span class="o">{</span>
<span class="n">enter</span><span class="o">(</span><span class="n">user</span><span class="o">.</span><span class="na">firstName</span><span class="o">,</span> <span class="n">into</span><span class="o">(</span><span class="s">"name"</span><span class="o">));</span>
<span class="n">enter</span><span class="o">(</span><span class="n">user</span><span class="o">.</span><span class="na">lastName</span><span class="o">,</span> <span class="n">into</span><span class="o">(</span><span class="s">"lastName"</span><span class="o">));</span>
<span class="n">enter</span><span class="o">(</span><span class="n">user</span><span class="o">.</span><span class="na">age</span><span class="o">,</span> <span class="n">into</span><span class="o">(</span><span class="s">"age"</span><span class="o">));</span>
<span class="n">enterMaritalStatus</span><span class="o">(</span><span class="n">user</span><span class="o">.</span><span class="na">isMarried</span><span class="o">);</span>
<span class="n">user</span><span class="o">.</span><span class="na">accomplishments</span><span class="o">.</span><span class="na">forEach</span><span class="o">(</span><span class="k">this</span><span class="o">::</span><span class="n">addAccomplishment</span><span class="o">);</span>
<span class="o">}</span>
</code></pre></div></div>
<p>I’d explain the pattern this way. If we have multiple objects which have some common logic (in the case above <code class="language-plaintext highlighter-rouge">createUser</code> accepts five parameters - first name, last name, age, marital status etc), it’s better to merge them in one entity. In this case <code class="language-plaintext highlighter-rouge">User</code> will be our Value Object, which aggregates all needed information about actual user into it.</p>
<p>Why do we call it “Value Object”? Because it’s immutable (it cannot be changed once it’s created) and its main purpose is to deliver data from point A to point B avoiding side effects including modifications and extensions.</p>
<p>To ease the process of creation of such objects you could use additional libraries, like <a href="https://projectlombok.org/">Lombok</a> in case you’re working with Java, or leverage built-in language features, like data classes in Kotlin. These tools will create needed constructors, generate getters for all fields and finalize them afterwards without any manual work from your side.</p>
<p>Data class example in Kotlin containing constructor and getters for all four parameters:</p>
<div class="language-kotlin highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">data class</span> <span class="nc">User</span><span class="p">(</span><span class="kd">val</span> <span class="py">firstName</span><span class="p">:</span> <span class="nc">String</span><span class="p">,</span>
<span class="kd">val</span> <span class="py">lastName</span><span class="p">:</span> <span class="nc">String</span><span class="p">,</span> <span class="kd">val</span> <span class="py">age</span><span class="p">:</span> <span class="nc">Int</span><span class="p">,</span>
<span class="kd">val</span> <span class="py">isMarried</span><span class="p">:</span> <span class="nc">Boolean</span><span class="p">,</span>
<span class="kd">val</span> <span class="py">accomplishments</span><span class="p">:</span> <span class="nc">List</span><span class="p"><</span><span class="nc">String</span><span class="p">>)</span>
</code></pre></div></div>
<h2 id="builder">Builder</h2>
<p>Let’s imaging we have extremely complex object and it can be configured in many different ways. The first option which comes to mind is to create as many types of constructors as we have parameter variations in this object and every time we meet new one we’d need to create new constructor for that. As a result we’ll get crazy amount of constructors which will definitely confuse the end-user of the framework.
Builder pattern is used to make the process of building such objects easier and with the help of modern IDE hints more intuitive.</p>
<p>Here is the example of Builder pattern usage:</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nc">Server</span> <span class="n">server</span> <span class="o">=</span> <span class="nc">Server</span><span class="o">.</span><span class="na">runtimeBuilder</span><span class="o">()</span>
<span class="o">.</span><span class="na">withUrl</span><span class="o">(</span><span class="s">"http://localhost"</span><span class="o">)</span>
<span class="o">.</span><span class="na">withPort</span><span class="o">(</span><span class="mi">1234</span><span class="o">)</span>
<span class="o">.</span><span class="na">withoutLogging</span><span class="o">()</span>
<span class="o">.</span><span class="na">withResponsesEnqueued</span><span class="o">(</span><span class="k">new</span> <span class="nc">Response</span><span class="o">())</span>
<span class="o">.</span><span class="na">build</span><span class="o">();</span>
</code></pre></div></div>
<p>At the beginning the special <code class="language-plaintext highlighter-rouge">ServerBuilder</code> object is created and after that it can be configured before actual <code class="language-plaintext highlighter-rouge">Server</code> object is built. That said only <code class="language-plaintext highlighter-rouge">build</code> method returns the actual <code class="language-plaintext highlighter-rouge">Server</code> object, other ones return the original <code class="language-plaintext highlighter-rouge">ServerBuilder</code> for further configuration.</p>
<p>Under the hood it would look something like this:</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">public</span> <span class="kd">class</span> <span class="nc">Server</span> <span class="o">{</span>
<span class="kd">private</span> <span class="kd">final</span> <span class="nc">String</span> <span class="n">url</span><span class="o">;</span>
<span class="kd">private</span> <span class="kd">final</span> <span class="kt">int</span> <span class="n">port</span><span class="o">;</span>
<span class="kd">private</span> <span class="kd">final</span> <span class="nc">List</span><span class="o"><</span><span class="nc">Response</span><span class="o">></span> <span class="n">responseQueue</span><span class="o">;</span>
<span class="kd">private</span> <span class="kd">final</span> <span class="kt">boolean</span> <span class="n">isLoggingEnabled</span><span class="o">;</span>
<span class="kd">private</span> <span class="nf">Server</span><span class="o">(</span><span class="nc">String</span> <span class="n">url</span><span class="o">,</span> <span class="kt">int</span> <span class="n">port</span><span class="o">,</span> <span class="nc">List</span><span class="o"><</span><span class="nc">Response</span><span class="o">></span> <span class="n">responseQueue</span><span class="o">,</span> <span class="kt">boolean</span> <span class="n">isLoggingEnabled</span><span class="o">)</span> <span class="o">{</span>
<span class="k">this</span><span class="o">.</span><span class="na">url</span> <span class="o">=</span> <span class="n">url</span><span class="o">;</span>
<span class="k">this</span><span class="o">.</span><span class="na">port</span> <span class="o">=</span> <span class="n">port</span><span class="o">;</span>
<span class="k">this</span><span class="o">.</span><span class="na">responseQueue</span> <span class="o">=</span> <span class="n">responseQueue</span><span class="o">;</span>
<span class="k">this</span><span class="o">.</span><span class="na">isLoggingEnabled</span> <span class="o">=</span> <span class="n">isLoggingEnabled</span><span class="o">;</span>
<span class="o">}</span>
<span class="kd">public</span> <span class="kd">static</span> <span class="nc">ServerBuilder</span> <span class="nf">runtimeBuilder</span><span class="o">()</span> <span class="o">{</span>
<span class="k">return</span> <span class="k">new</span> <span class="nf">ServerBuilder</span><span class="o">();</span>
<span class="o">}</span>
<span class="kd">public</span> <span class="kd">static</span> <span class="kd">class</span> <span class="nc">ServerBuilder</span> <span class="o">{</span>
<span class="kd">private</span> <span class="nc">String</span> <span class="n">url</span> <span class="o">=</span> <span class="s">"http://10.0.0.0"</span>
<span class="kd">private</span> <span class="kd">final</span> <span class="kt">int</span> <span class="n">port</span> <span class="o">=</span> <span class="mi">6767</span><span class="o">;</span>
<span class="kd">private</span> <span class="kd">final</span> <span class="nc">List</span><span class="o"><</span><span class="nc">Response</span><span class="o">></span> <span class="n">responseQueue</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">ArrayList</span><span class="o"><>();</span>
<span class="kd">private</span> <span class="kd">final</span> <span class="kt">boolean</span> <span class="n">isLoggingEnabled</span> <span class="o">=</span> <span class="kc">true</span><span class="o">;</span>
<span class="kd">public</span> <span class="nc">Server</span> <span class="nf">build</span><span class="o">()</span> <span class="o">{</span>
<span class="k">return</span> <span class="k">new</span> <span class="nf">Server</span><span class="o">(</span><span class="n">url</span><span class="o">,</span> <span class="n">port</span><span class="o">,</span> <span class="n">responseQueue</span><span class="o">,</span> <span class="n">isLoggingEnabled</span><span class="o">);</span>
<span class="o">}</span>
<span class="kd">public</span> <span class="nc">ServerBuilder</span> <span class="nf">withUrl</span><span class="o">(</span><span class="nc">String</span> <span class="n">url</span><span class="o">)</span> <span class="o">{</span>
<span class="k">this</span><span class="o">.</span><span class="na">url</span> <span class="o">=</span> <span class="n">url</span><span class="o">;</span>
<span class="k">return</span> <span class="k">this</span><span class="o">;</span>
<span class="o">}</span>
<span class="kd">public</span> <span class="nc">ServerBuilder</span> <span class="nf">withPort</span><span class="o">(</span><span class="kt">int</span> <span class="n">port</span><span class="o">)</span> <span class="o">{</span>
<span class="k">this</span><span class="o">.</span><span class="na">port</span> <span class="o">=</span> <span class="n">port</span><span class="o">;</span>
<span class="k">return</span> <span class="k">this</span><span class="o">;</span>
<span class="o">}</span>
<span class="kd">public</span> <span class="nc">ServerBuilder</span> <span class="nf">withResponsesEnqueued</span><span class="o">(</span><span class="nc">Response</span><span class="o">...</span>
<span class="n">responses</span><span class="o">)</span> <span class="o">{</span>
<span class="k">this</span><span class="o">.</span><span class="na">responseQueue</span><span class="o">.</span><span class="na">clear</span><span class="o">();</span>
<span class="k">for</span> <span class="o">(</span><span class="nc">Response</span> <span class="n">response</span> <span class="o">:</span> <span class="n">responses</span><span class="o">)</span> <span class="o">{</span>
<span class="k">this</span><span class="o">.</span><span class="na">responseQueue</span><span class="o">.</span><span class="na">add</span><span class="o">(</span><span class="n">response</span><span class="o">);</span>
<span class="o">}</span>
<span class="k">return</span> <span class="k">this</span><span class="o">;</span>
<span class="o">}</span>
<span class="kd">public</span> <span class="nc">ServerBuilder</span> <span class="nf">withoutLogging</span><span class="o">()</span> <span class="o">{</span>
<span class="k">this</span><span class="o">.</span><span class="na">isLoggingEnabled</span> <span class="o">=</span> <span class="kc">false</span><span class="o">;</span>
<span class="k">return</span> <span class="k">this</span><span class="o">;</span>
<span class="o">}</span>
<span class="o">}</span>
<span class="o">...</span>
<span class="o">}</span>
<span class="o">}</span>
</code></pre></div></div>
<p class="notice--info">In the example above all the parameters have default values when <code class="language-plaintext highlighter-rouge">ServerBuilder</code> is created, which means user can create <code class="language-plaintext highlighter-rouge">Server</code> object without having to set up anything. This might be changed as well if we want our object to have mandatory configurable fields. In that case I’d throw exception saying that object is not fully configured.</p>
<h2 id="assert-objectmatchers">Assert Object/Matchers</h2>
<p>Majority of the people heard about the next pattern but I’ve seen only few actually using it. Its name is “Assert Object” or simply Matcher. Usually it might be used whenever we need to do domain specific assertions on some object. Let’s take a look at the example below:</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nd">@Test</span>
<span class="kd">public</span> <span class="kt">void</span> <span class="nf">onlyOneResponseWithErrorCodeShouldBeReturned</span><span class="o">()</span> <span class="o">{</span>
<span class="nc">List</span><span class="o"><</span><span class="nc">Response</span><span class="o">></span> <span class="n">responses</span> <span class="o">=</span> <span class="n">server</span><span class="o">.</span><span class="na">getResponses</span><span class="o">();</span>
<span class="n">assertEquals</span><span class="o">(</span><span class="mi">1</span><span class="o">,</span> <span class="n">responses</span><span class="o">.</span><span class="na">size</span><span class="o">());</span>
<span class="nc">Response</span> <span class="n">response</span> <span class="o">=</span> <span class="n">responses</span><span class="o">.</span><span class="na">get</span><span class="o">(</span><span class="mi">0</span><span class="o">);</span>
<span class="n">assertEquals</span><span class="o">(</span><span class="nc">ResponseCode</span><span class="o">.</span><span class="na">ERROR_403</span><span class="o">,</span> <span class="n">response</span><span class="o">.</span><span class="na">getCode</span><span class="o">());</span>
<span class="o">}</span>
</code></pre></div></div>
<figure class="">
<img src="/assets/images/arrow.png" alt="" /></figure>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nd">@Test</span>
<span class="kd">public</span> <span class="kt">void</span> <span class="nf">onlyOneResponseWithErrorCodeShouldBeReturned</span><span class="o">()</span> <span class="o">{</span>
<span class="n">assertThat</span><span class="o">(</span><span class="n">server</span><span class="o">).</span><span class="na">hadSingleResponseWithCode</span><span class="o">(</span>
<span class="nc">ResponseCode</span><span class="o">.</span><span class="na">ERROR_403</span><span class="o">);</span>
<span class="o">}</span>
<span class="kd">public</span> <span class="kd">static</span> <span class="nc">ServerAssert</span> <span class="nf">assertThat</span><span class="o">(</span><span class="nc">Server</span> <span class="n">server</span><span class="o">)</span> <span class="o">{</span>
<span class="k">return</span> <span class="k">new</span> <span class="nf">ServerAssert</span><span class="o">(</span><span class="n">server</span><span class="o">);</span>
<span class="o">}</span>
<span class="kd">public</span> <span class="kd">class</span> <span class="nc">ServerAssert</span> <span class="o">{</span>
<span class="kd">private</span> <span class="kd">final</span> <span class="nc">Server</span> <span class="n">server</span><span class="o">;</span>
<span class="kd">public</span> <span class="nf">ServerAssert</span><span class="o">(</span><span class="nc">Server</span> <span class="n">server</span><span class="o">)</span> <span class="o">{</span>
<span class="k">this</span><span class="o">.</span><span class="na">server</span> <span class="o">=</span> <span class="n">server</span><span class="o">;</span>
<span class="o">}</span>
<span class="kd">public</span> <span class="kt">void</span> <span class="nf">hadSingleResponseWithCode</span><span class="o">(</span><span class="nc">ResponseCode</span>
<span class="n">responseCode</span><span class="o">)</span> <span class="o">{</span>
<span class="nc">List</span><span class="o"><</span><span class="nc">Response</span><span class="o">></span> <span class="n">responses</span> <span class="o">=</span> <span class="n">server</span><span class="o">.</span><span class="na">getResponses</span><span class="o">();</span>
<span class="n">assertEquals</span><span class="o">(</span><span class="mi">1</span><span class="o">,</span> <span class="n">responses</span><span class="o">.</span><span class="na">size</span><span class="o">());</span>
<span class="nc">Response</span> <span class="n">response</span> <span class="o">=</span> <span class="n">responses</span><span class="o">.</span><span class="na">get</span><span class="o">(</span><span class="mi">0</span><span class="o">);</span>
<span class="n">assertEquals</span><span class="o">(</span><span class="n">responseCode</span><span class="o">,</span> <span class="n">response</span><span class="o">.</span><span class="na">getCode</span><span class="o">());</span>
<span class="o">}</span>
<span class="o">}</span>
</code></pre></div></div>
<p>In the original example we do not see obvious logic of the checks in the test. It might be not clear to the user what exactly is going to be verified. At first we’re validating that server returns only one response and then that the response has 403 error code. We have to split these checks to two separate assertions because otherwise it would be difficult to understand what went wrong. But for someone not familiar with our domain it would be still difficult to understand test like that.</p>
<p>Matcher pattern helps us to create asserts as reusable constructions, which reduce overall code duplication. For instance, if we need to verify that server returns one response, but with 200 response code.</p>
<p>Besides that this approach creates domain logic in our tests. That’s why I prefer to implement it the way shown above, when we create group of asserts and put them together in separate class (e.g. <code class="language-plaintext highlighter-rouge">ServerAssert</code>), which is responsible for all possible checks on the <code class="language-plaintext highlighter-rouge">Server</code> object. Then all we need to do is to create static method <code class="language-plaintext highlighter-rouge">assertThat</code> accepting <code class="language-plaintext highlighter-rouge">Server</code> object and returning <code class="language-plaintext highlighter-rouge">ServerAssert</code> instead. It looks great and can be read much easier than before and the assertion code underneath stays the same.</p>
<p class="notice--info">Another option for creating matchers is to build static methods for each of them. There are multiple libraries out there which already have a lot of ready-to-use matchers bundled in and provide easy-to-use API for creating your own ones. The most popular of them are <a href="http://hamcrest.org/">Hamcrest</a> and <a href="http://joel-costigliola.github.io/assertj/">AssertJ</a>. If you haven’t used them before I suggest at least to pay attention to those ones and think about building them into your framework. Again, they won’t have your domain specific Matchers, but they might significantly simplify creating ones.</p>
<h2 id="data-registry">Data Registry</h2>
<p>The next pattern is interesting one. The main approach is as follows: we want our tests to be independent and try to split our test data across them, but as a result we get completely opposite. For example, test A uses user1, user2 and user3 and they are hardcoded as a test data. This might be the problem since we want completely independent tests, right? But we force other test to be aware that user1, user2 and user3 are already occupied by test A. Another concern is that developer not familiar with this could use those users in other tests and this may cause the issues.</p>
<p>Data registry allows us to generate unique data and avoid duplications. In the example below I’m using the simplest approach possible: on every <code class="language-plaintext highlighter-rouge">getUser</code> invocation static thread-safe counter will be incremented by 1, guaranteeing that each time unique user is created.</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">public</span> <span class="kd">static</span> <span class="kd">class</span> <span class="nc">UserRegistry</span> <span class="o">{</span>
<span class="kd">private</span> <span class="kd">static</span> <span class="nc">AtomicInteger</span> <span class="no">COUNTER</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">AtomicInteger</span><span class="o">(</span><span class="mi">0</span><span class="o">);</span>
<span class="kd">public</span> <span class="kd">static</span> <span class="nf">getUser</span><span class="o">()</span> <span class="o">{</span>
<span class="kt">int</span> <span class="n">index</span> <span class="o">=</span> <span class="no">COUNTER</span><span class="o">.</span><span class="na">incrementAndGet</span><span class="o">();</span>
<span class="k">return</span> <span class="k">new</span> <span class="nf">User</span><span class="o">(</span><span class="s">"User_"</span> <span class="o">+</span> <span class="n">index</span><span class="o">,</span> <span class="n">index</span><span class="o">);</span>
<span class="o">}</span>
<span class="o">}</span>
</code></pre></div></div>
<p>In your case the pattern logic might be much more complex, e.g. the registry could take the user from database, file, predefined data set etc. But the outcome will be the same: your tests will be truly independent, since each time they use UserRegistry they get exclusive user avoiding test interception issues.</p>
<h2 id="object-pool--flyweight">Object Pool / Flyweight</h2>
<p>The next pattern is used by even less developers. Flyweight is the classic pattern from GoF book, which solves the problem of retaining and operating with the heavy in terms of resources objects or set of objects. Instead of creating them every time we need them, we take them, use them and return them to so called Pool for future uses.</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">private</span> <span class="kd">final</span> <span class="nc">UserPool</span> <span class="no">USER_POOL</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">UserPool</span><span class="o">();</span>
<span class="kd">private</span> <span class="nc">User</span> <span class="n">user</span><span class="o">;</span>
<span class="nd">@Before</span>
<span class="kd">public</span> <span class="kt">void</span> <span class="nf">setUp</span><span class="o">()</span> <span class="o">{</span>
<span class="n">user</span> <span class="o">=</span> <span class="no">USER_POOL</span><span class="o">.</span><span class="na">getAvailableUser</span><span class="o">();</span>
<span class="o">}</span>
<span class="nd">@Test</span>
<span class="kd">public</span> <span class="kt">void</span> <span class="nf">userShouldBeAbleToLogin</span><span class="o">()</span> <span class="o">{</span>
<span class="nc">HomePage</span> <span class="n">homePage</span> <span class="o">=</span> <span class="n">loginPage</span><span class="o">().</span><span class="na">loginAs</span><span class="o">(</span><span class="n">user</span><span class="o">);</span>
<span class="n">assertThat</span><span class="o">(</span><span class="n">homePage</span><span class="o">.</span><span class="na">getUsername</span><span class="o">(),</span> <span class="n">is</span><span class="o">(</span><span class="n">user</span><span class="o">.</span><span class="na">getName</span><span class="o">()))</span>
<span class="o">}</span>
<span class="nd">@After</span>
<span class="kd">public</span> <span class="kt">void</span> <span class="nf">tearDown</span><span class="o">()</span> <span class="o">{</span>
<span class="no">USER_POOL</span><span class="o">.</span><span class="na">releaseUser</span><span class="o">(</span><span class="n">user</span><span class="o">);</span>
<span class="o">}</span>
</code></pre></div></div>
<p>Using this pattern we could implement lots of interesting things, for instance, browser pool. I heard quiet a few complains from different people that web-tests take crazy amounts of time, because they require to start browser, load first page, import user profiles etc. But it’s not necessarily to create browser in the actual test, instead we could use Background Pool, which set up to retain needed amount of “hot” browsers. After we’re done with the browser we just return it to the pool and clear its data. And this might be done in background, in parallel with the actual test threads. And only after browser is ready to be used again it can be given back to the next test as a fresh instance.</p>
<p>That said, this pool configuration and browser set up parts might be taken out outside of the test, significantly minimizing the time and resources spent on it.</p>
<p>The other example is the page usage. You don’t have to wait until the needed page is opened, if all of the tests start from the same page. You could have the Page Pool as well and request it from there. This means it’s gonna be opened in one of the browser instances in background beforehand and will be waiting for the test to pick it up in ready-to-use state.</p>
<p>Another well-known option for using this approach would be Database Pool. Instead of working with the real database, we could start the needed number of database containers on the different ports (it can be done with Docker or other virtualization tool) and “kill” it after we do not need it anymore. That way we’d always have clean database without the need of tearing it down, cleaning it up, collection and uploading the data etc.</p>
<h2 id="data-provider">Data Provider</h2>
<p>Data Provider is one of the most widely used data patterns among test engineers. If you want to implement Data-Driven tests and are willing to run the same test logic on multiple sets of data, you could load the data from outer sources (like Excel or CVS table), remote services or hardcode them in-place.</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nd">@DataProvider</span>
<span class="kd">private</span> <span class="kd">static</span> <span class="nc">Object</span><span class="o">[][]</span> <span class="nf">testDataProvider</span><span class="o">()</span> <span class="o">{</span>
<span class="k">try</span> <span class="o">{</span>
<span class="k">return</span> <span class="nc">ReadExcelSheet</span><span class="o">.</span><span class="na">getTableArray</span><span class="o">(</span>
<span class="s">"src/main/resources/TestData.xls"</span><span class="o">);</span>
<span class="o">}</span> <span class="k">catch</span> <span class="o">(</span><span class="nc">Exception</span> <span class="n">e</span><span class="o">)</span> <span class="o">{</span>
<span class="k">return</span> <span class="kc">null</span><span class="o">;</span>
<span class="o">}</span>
<span class="o">}</span>
</code></pre></div></div>
<p>This could be done in the way I showed above by reading from source and returning untyped data (simple array of arrays or strings). But modern approach would be to utilize <a href="https://alexilyenko.github.io/patterns-2/#value-object">Value Object pattern</a>, we were talking about previously, and provide data in terms of entities.</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nd">@DataProvider</span>
<span class="kd">private</span> <span class="kd">static</span> <span class="nc">Iterator</span><span class="o"><</span><span class="nc">Object</span><span class="o">[]></span> <span class="nf">devices</span><span class="o">()</span> <span class="o">{</span>
<span class="k">return</span> <span class="nf">asList</span><span class="o">(</span>
<span class="k">new</span> <span class="nc">Object</span><span class="o">[]</span> <span class="o">{</span> <span class="k">new</span> <span class="nc">Device</span><span class="o">(</span><span class="s">"iPhone X"</span><span class="o">,</span> <span class="nc">Platform</span><span class="o">.</span><span class="na">IOS</span><span class="o">,</span> <span class="s">"11.2"</span><span class="o">)</span> <span class="o">},</span>
<span class="k">new</span> <span class="nc">Object</span><span class="o">[]</span> <span class="o">{</span> <span class="k">new</span> <span class="nc">Device</span><span class="o">(</span><span class="s">"iPhone 7 Plus"</span><span class="o">,</span> <span class="nc">Platform</span><span class="o">.</span><span class="na">IOS</span><span class="o">,</span> <span class="s">"10.3"</span><span class="o">)</span> <span class="o">},</span>
<span class="k">new</span> <span class="nc">Object</span><span class="o">[]</span> <span class="o">{</span> <span class="k">new</span> <span class="nc">Device</span><span class="o">(</span><span class="s">"Google Pixel 2"</span><span class="o">,</span> <span class="nc">Platform</span><span class="o">.</span><span class="na">ANDROID</span><span class="o">,</span> <span class="s">"8.0"</span><span class="o">)</span> <span class="o">}</span>
<span class="o">).</span><span class="na">iterator</span><span class="o">();</span>
<span class="o">}</span>
<span class="nd">@Test</span><span class="o">(</span><span class="n">dataProvider</span> <span class="o">=</span> <span class="s">"devices"</span><span class="o">)</span>
<span class="kd">public</span> <span class="kt">void</span> <span class="nf">pageShouldBeOpenedOnDevice</span><span class="o">(</span><span class="nc">Device</span> <span class="n">device</span><span class="o">)</span> <span class="o">{</span>
<span class="n">startDevice</span><span class="o">(</span><span class="n">device</span><span class="o">);</span>
<span class="c1">// some test steps</span>
<span class="o">}</span>
</code></pre></div></div>
<p>You need to mark your method with the annotation <code class="language-plaintext highlighter-rouge">dataProvider</code> or make it parametrized with the help of <a href="https://junit.org/junit5/docs/current/user-guide/#writing-tests-parameterized-tests">JUnit</a> and make it return the data set for using it in the test. The rest (parsing and iterating) will be done by the framework, all you have to do is to use data as you would use in regular test.</p>
<p class="notice--info">It’s important that we used both patterns (Data Provider and Value Object) in one approach, since it helped us to avoid the passing of multiple parameters to method and to make code cleaner and more readable.</p>
<p>I love using Data Patterns in my automation, they help me to keep my code healthy and handle resources in the most optimized way possible. In case, you’re interested in other patterns which might help you in writing clean and robust tests, check out my other posts:</p>
<ul>
<li><a href="https://alexilyenko.github.io/patterns-1/">Design Patterns in Test Automation I</a></li>
<li><a href="https://alexilyenko.github.io/patterns-3/">Design Patterns in Test Automation III</a></li>
</ul>
<p><a href="https://alexilyenko.github.io/"><img src="https://alexilyenko.github.io/assets/images/share_message.png" alt="Feel free to share!" /></a></p>Alex IlyenkoData Patterns in Test Automation and DevelopmentDesign Patterns in Test Automation I2017-11-13T00:00:00-05:002017-11-13T00:00:00-05:00https://alexilyenko.github.io/patterns-1<h2 id="whats-design-pattern">What’s design pattern</h2>
<p>There is no such thing as “good” or “bad” design pattern. First of all the term “design pattern” was introduced to represent the problem and its optimal solution. That’s why if you don’t want to use the particular design pattern in development, test automation, etc., it doesn’t mean it’s bad, wrong or outdated. It just means that the pattern is not applicable to your problem. Otherwise if this pattern resolves similar problem to yours, it’s better to take a closer look at it.</p>
<p>Thus this is generally not a good idea to use the pattern only because you heard “it’s cool” from someone. Instead you need to understand its goals, problems it can solve and benefits you might get from using it.</p>
<figure class="">
<img src="/assets/images/patterns.png" alt="GoF Patterns Classification" /><figcaption>
GoF Patterns Classification
</figcaption></figure>
<p>There has been always a lot of problem in designing software architecture, faced by different people. Some of them came up with the solutions to them and created the patterns. The first ones who did that and classified most of the classic design patterns were the <a href="https://en.wikipedia.org/wiki/Design_Patterns">Gang of Four</a>. They published the book “Design Patterns”, where they showed why, when and how to use every particular known pattern in object oriented programming. Some of the patterns, described in the book, are outdated, some are missing, but the main concept is still the same - if there is a problem, there should be a solution to it.</p>
<p>This idea was so great, almost every industry adopted it. Software test automation was not an exception. Today we will talk ones, which are specific to testing and automation.</p>
<h2 id="why-do-we-need-patterns-in-our-tests">Why do we need patterns in our tests</h2>
<p>The main reasons for leveraging design patterns in test automation is increasing of stability, maintainability, flexibility, reliability and clarity. All of them are important but different aspect can be reached by using different patterns.</p>
<p>Most of them could be assigned to some group by the problem it helps to solve:</p>
<ul>
<li>Structural patterns</li>
<li>Data patterns</li>
<li>Technical patterns</li>
<li>Business involvement patterns</li>
</ul>
<p>In the first part I’ll cover structural patterns, the next articles will cover others.</p>
<h2 id="structural-patterns">Structural Patterns</h2>
<p>The main goal of structural patterns is to structure test code to simplify maintenance, avoid duplication and increase clarity. By doing that we will make it easier for other test engineers, not familiar with other code base, to understand and to start working with the tests right away.</p>
<h2 id="page-object">Page Object</h2>
<figure class="">
<img src="/assets/images/page_object_scheme_xctest.jpeg" alt="Page Object example in iOS tests" /><figcaption>
Page Object example in iOS tests
</figcaption></figure>
<p>The first pattern we’ll take a look at is Page Object. Probably it’s the most famous one among test engineers. The main problem it solves is the separation of technical details (e.g. user interface elements on the page/screen) and actual test logic of UI test.</p>
<p>Secondly, it helps to reduce code duplication. We would like to reuse our code in different test scripts and Page Object may help us to do that. Sometimes when I add new tests to my projects I don’t have to write any extra class or function except of the test script. Because they were already written with Page Object!</p>
<p>And finally Page Object makes tests more readable and understandable. It shows which page is user on right now and prevents test from doing actions not related to current page.</p>
<p>Of course, Page Object is not a panacea. If I had small amount of tests in my project (i.e. less than 20) and did not plan to extend those, I would consider not to use it. Just because the effort to create page objects would not pay off. Each pattern may generate a great value for you project but they should not be your goal in favor of project needs.</p>
<p>I wrote a couple of posts on Page Object in <a href="https://alexilyenko.github.io/xcuitest-page-object/">iOS</a> and <a href="https://alexilyenko.github.io/uiautomator-page-object/">Android</a> testing. If you’re interested in technical details, don’t hesitate to check them out.</p>
<h2 id="fluent--chain-of-invocations">Fluent / Chain of Invocations</h2>
<p>The second pattern we were going to talk about is Chain of Invocations. It’s usually being used along with Page Object, so most of you might be familiar with it too. The problem it resolves is helping test developer to determine if she can use the object or she should switch to other one. For example, user is on the login page and he’s pressing some button. By writing code in old fashion we wouldn’t be sure if he’s still on the login page or already on the home page.</p>
<p>And now just imagine if you had 50 similar methods in your test. You cannot be sure if you can invoke them right away, or they depend on some order, or even they can not be invoked after one of them was executed. For example, can I click on hint dialog if I haven’t entered any character into input field? Probably not, because at the very moment, it doesn’t exist yet!</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">public</span> <span class="kt">void</span> <span class="nf">loginToAccountShouldWork</span><span class="o">()</span> <span class="o">{</span>
<span class="n">homePage</span><span class="o">.</span><span class="na">typeLogin</span><span class="o">(</span><span class="n">user</span><span class="o">.</span><span class="na">login</span><span class="o">);</span>
<span class="n">homePage</span><span class="o">.</span><span class="na">typePassword</span><span class="o">(</span><span class="n">user</span><span class="o">.</span><span class="na">password</span><span class="o">);</span>
<span class="n">homePage</span><span class="o">.</span><span class="na">tapSignInButton</span><span class="o">();</span>
<span class="n">assertThat</span><span class="o">(</span><span class="n">homePage</span><span class="o">.</span><span class="na">isSignedIn</span><span class="o">(</span><span class="n">user</span><span class="o">));</span>
<span class="o">}</span>
</code></pre></div></div>
<figure class="">
<img src="/assets/images/arrow.png" alt="" /></figure>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">public</span> <span class="kt">void</span> <span class="nf">loginToAccountShouldWork</span><span class="o">()</span> <span class="o">{</span>
<span class="n">homePage</span>
<span class="o">.</span><span class="na">typeLogin</span><span class="o">(</span><span class="n">user</span><span class="o">.</span><span class="na">login</span><span class="o">)</span>
<span class="o">.</span><span class="na">typePassword</span><span class="o">(</span><span class="n">user</span><span class="o">.</span><span class="na">password</span><span class="o">)</span>
<span class="o">.</span><span class="na">tapSignInButton</span><span class="o">();</span>
<span class="n">assertThat</span><span class="o">(</span><span class="n">homePage</span><span class="o">.</span><span class="na">isSignedIn</span><span class="o">(</span><span class="n">user</span><span class="o">));</span>
<span class="o">}</span>
</code></pre></div></div>
<p>This pattern is important in case you’re trying to scale your test automation framework. Large amount of pages, elements and methods available for usage may create a confusion, especially for someone not really familiar with your domain. But by implementing Fluent Invocations modern IDE will give you a hint every time you’ll try to invoke some method on the object by autocompletion feature.</p>
<p>Usually if you want to interrupt method chain, all you have to do is to assign the return value (e.g. string, number etc.) to some variable. This way you’d show that this is the end of the sequence of invocations. At this point developer should stop and think about next method to be invoked in test script before doing that.</p>
<p>Chain of Invocations is easy to implement. All you have to do is to return the value in every method of Page Object. This might be <code class="language-plaintext highlighter-rouge">this</code>, some value or any other object, for example next Page Object after method invocation (transition to other page).</p>
<p>This pattern doesn’t help to reduce code a lot (that’s not its mission thought), but it allows you to not repeat yourself by putting the object again and again before invoking its methods. Also IMO it makes code a little bit prettier.</p>
<h2 id="page-factory">Page Factory</h2>
<p>Page Factory is an extension to Page Object pattern. It helps to encapsulate page’s attributes and methods even more by providing <code class="language-plaintext highlighter-rouge">FindBy</code> annotations.</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">public</span> <span class="kd">class</span> <span class="nc">LoginPage</span> <span class="kd">extends</span> <span class="nc">BasePage</span> <span class="o">{</span>
<span class="kd">private</span> <span class="kd">static</span> <span class="kd">final</span> <span class="nc">By</span> <span class="no">USERNAME_FIELD</span> <span class="o">=</span>
<span class="nc">By</span><span class="o">.</span><span class="na">id</span><span class="o">(</span><span class="s">"usernameField"</span><span class="o">);</span>
<span class="kd">private</span> <span class="kd">static</span> <span class="kd">final</span> <span class="nc">By</span> <span class="no">PASSWORD_FIELD</span> <span class="o">=</span>
<span class="nc">By</span><span class="o">.</span><span class="na">id</span><span class="o">(</span><span class="s">"passwordField"</span><span class="o">);</span>
<span class="kd">private</span> <span class="kd">static</span> <span class="kd">final</span> <span class="nc">By</span> <span class="no">LOGIN_BUTTON</span> <span class="o">=</span>
<span class="nc">By</span><span class="o">.</span><span class="na">id</span><span class="o">(</span><span class="s">"loginButton"</span><span class="o">);</span>
<span class="kd">public</span> <span class="nf">LoginPage</span><span class="o">(</span><span class="nc">WebDriver</span> <span class="n">driver</span><span class="o">)</span> <span class="o">{</span>
<span class="kd">super</span><span class="o">(</span><span class="n">driver</span><span class="o">);</span>
<span class="o">}</span>
<span class="kd">public</span> <span class="nc">HomePage</span> <span class="nf">loginAs</span><span class="o">(</span><span class="nc">User</span> <span class="n">user</span><span class="o">)</span> <span class="o">{</span>
<span class="n">driver</span><span class="o">.</span><span class="na">findElement</span><span class="o">(</span><span class="no">USERNAME_FIELD</span><span class="o">)</span>
<span class="o">.</span><span class="na">sendKeys</span><span class="o">(</span><span class="n">user</span><span class="o">.</span><span class="na">username</span><span class="o">);</span>
<span class="n">driver</span><span class="o">.</span><span class="na">findElement</span><span class="o">(</span><span class="no">PASSWORD_FIELD</span><span class="o">)</span>
<span class="o">.</span><span class="na">sendKeys</span><span class="o">(</span><span class="n">user</span><span class="o">.</span><span class="na">password</span><span class="o">);</span>
<span class="n">driver</span><span class="o">.</span><span class="na">findElement</span><span class="o">(</span><span class="no">LOGIN_BUTTON</span><span class="o">)</span>
<span class="o">.</span><span class="na">click</span><span class="o">();</span>
<span class="k">return</span> <span class="k">new</span> <span class="nf">HomePage</span><span class="o">(</span><span class="n">driver</span><span class="o">);</span>
<span class="o">}</span>
<span class="o">}</span>
</code></pre></div></div>
<figure class="">
<img src="/assets/images/arrow.png" alt="" /></figure>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">public</span> <span class="kd">class</span> <span class="nc">LoginPage</span> <span class="kd">extends</span> <span class="nc">BasePage</span> <span class="o">{</span>
<span class="nd">@FindBy</span>
<span class="kd">private</span> <span class="nc">WebElement</span> <span class="n">usernameField</span><span class="o">;</span>
<span class="nd">@FindBy</span>
<span class="kd">private</span> <span class="nc">WebElement</span> <span class="n">passwordField</span><span class="o">;</span>
<span class="nd">@FindBy</span>
<span class="kd">private</span> <span class="nc">WebElement</span> <span class="n">loginButton</span><span class="o">;</span>
<span class="kd">public</span> <span class="nf">LoginPage</span><span class="o">(</span><span class="nc">WebDriver</span> <span class="n">driver</span><span class="o">)</span> <span class="o">{</span>
<span class="kd">super</span><span class="o">(</span><span class="n">driver</span><span class="o">);</span>
<span class="nc">PageFactory</span><span class="o">.</span><span class="na">initElements</span><span class="o">(</span><span class="n">driver</span><span class="o">,</span> <span class="k">this</span><span class="o">);</span>
<span class="o">}</span>
<span class="kd">public</span> <span class="nc">HomePage</span> <span class="nf">loginAs</span><span class="o">(</span><span class="nc">User</span> <span class="n">user</span><span class="o">)</span> <span class="o">{</span>
<span class="n">usernameField</span><span class="o">.</span><span class="na">sendKeys</span><span class="o">(</span><span class="n">user</span><span class="o">.</span><span class="na">username</span><span class="o">);</span>
<span class="n">passwordField</span><span class="o">.</span><span class="na">sendKeys</span><span class="o">(</span><span class="n">user</span><span class="o">.</span><span class="na">password</span><span class="o">);</span>
<span class="n">loginButton</span><span class="o">.</span><span class="na">click</span><span class="o">();</span>
<span class="k">return</span> <span class="k">new</span> <span class="nf">HomePage</span><span class="o">(</span><span class="n">driver</span><span class="o">);</span>
<span class="o">}</span>
<span class="o">}</span>
</code></pre></div></div>
<p>Pay attentions to <code class="language-plaintext highlighter-rouge">PageFactory#initElements</code> invocation. This static helper initializes all fields with <code class="language-plaintext highlighter-rouge">FindBy</code> annotations on the page, which will be found on it on each call. The main advantage is the fact that now we work directly with fields, buttons, windows etc. and do not worry about low level driver’s interactions exactly the same way our app users do.</p>
<h2 id="composition-of-page-elements">Composition of Page Elements</h2>
<figure class="">
<img src="/assets/images/components.png" alt="Breaking down page into Page Elements" /><figcaption>
Breaking down page into Page Elements
</figcaption></figure>
<p>Any web, desktop or mobile application consists of repeatable elements, and logic of their usage should be implemented again and again in our tests. For instance, every menu has list of links, every table has rows and columns, every form has input fields. In real life when we work with those elements we do not separate those components from the main element consisting them.</p>
<p>Thanks to Composition we could implement some elements once and reuse them every time we need them. Thus it helps to avoid code duplication by composing different web elements into widgets (high-level elements), like tables, menus, forms. This significantly reduces costs of extending and scaling of test automation framework when, for example new Page Objects needs to be created.</p>
<h2 id="loadable-component">Loadable Component</h2>
<p>This pattern is used by most of the developers who works with user interface tests. The thing is, when test makes the transition from one page to another, it doesn’t know if the targeted page was loaded completely. Obviously, either regular <code class="language-plaintext highlighter-rouge">sleep</code> or not waiting at all is not a solution here. So how do we solve this problem? Usually test engineers create explicit wait in constructor of the page class or in its ancestor and override if it’s needed.</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">public</span> <span class="kd">class</span> <span class="nc">HomePage</span> <span class="kd">extends</span> <span class="nc">BasPage</span> <span class="o">{</span>
<span class="nd">@FindBy</span><span class="o">(</span><span class="n">id</span> <span class="o">=</span> <span class="s">"someId"</span><span class="o">)</span>
<span class="kd">private</span> <span class="nc">WebElement</span> <span class="n">element</span><span class="o">;</span>
<span class="kd">public</span> <span class="nf">HomePage</span><span class="o">(</span><span class="nc">WebDriver</span> <span class="n">driver</span><span class="o">)</span> <span class="o">{</span>
<span class="kd">super</span><span class="o">(</span><span class="n">driver</span><span class="o">);</span>
<span class="nc">PageFactory</span><span class="o">.</span><span class="na">initElements</span><span class="o">(</span><span class="n">driver</span><span class="o">,</span> <span class="k">this</span><span class="o">);</span>
<span class="n">waitForElement</span><span class="o">(</span><span class="n">element</span><span class="o">);</span>
<span class="o">}</span>
<span class="o">}</span>
</code></pre></div></div>
<p>Again, there is not strict rule how to do this wait. Personally I like to move this waiting to <code class="language-plaintext highlighter-rouge">BasePage</code> and override abstract method for waiting particular element. How to do that in your case, completely up to you. But before implementing such complicated logic you need to be sure that your app page transition could cause you some problem. Otherwise just usual implicit wait might be enough. If you’re interested in implementation of explicit waits in your UI mobile tests and do not know where to start, I recommend to read my post about explicit waits in <a href="https://alexilyenko.github.io/uiautomator-waiting/">Android</a> or <a href="https://alexilyenko.github.io/xcuitest-waiting/">iOS</a>.</p>
<h2 id="strategy">Strategy</h2>
<p>Strategy pattern is used whenever we want to have more than one implementations of the same action/sequence of actions, which is done differently. Depending on the context we could choose the implementation.</p>
<p>The easiest example is user registration. You might want to have two different implementations of this particular action. The first one would be the actual flow of transitions through the pages for successful user registration. The other one would be short API call which is invoked when new user is needed for the test.</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">interface</span> <span class="nc">UserRegistrationStrategy</span> <span class="o">{</span>
<span class="nc">User</span> <span class="nf">register</span><span class="o">();</span>
<span class="o">}</span>
<span class="kd">class</span> <span class="nc">WebUserRegistrationStrategy</span>
<span class="kd">implements</span> <span class="nc">UserRegistrationStrategy</span> <span class="o">{</span>
<span class="nd">@Override</span>
<span class="kd">public</span> <span class="nc">User</span> <span class="nf">register</span><span class="o">()</span> <span class="o">{</span>
<span class="nc">String</span> <span class="n">username</span> <span class="o">=</span> <span class="nc">UserRegistry</span><span class="o">.</span><span class="na">getUsername</span><span class="o">();</span>
<span class="nc">String</span> <span class="n">password</span> <span class="o">=</span> <span class="nc">PasswordGenerator</span><span class="o">.</span><span class="na">generatePassword</span><span class="o">();</span>
<span class="nc">SignInPage</span><span class="o">.</span><span class="na">open</span><span class="o">()</span>
<span class="o">.</span><span class="na">pressSignUpButton</span><span class="o">()</span>
<span class="o">.</span><span class="na">enterUsername</span><span class="o">(</span><span class="n">username</span><span class="o">)</span>
<span class="o">.</span><span class="na">enterPassword</span><span class="o">(</span><span class="n">password</span><span class="o">)</span>
<span class="o">.</span><span class="na">clickRegisterButton</span><span class="o">();</span>
<span class="k">return</span> <span class="k">new</span> <span class="nf">User</span><span class="o">(</span><span class="n">username</span><span class="o">,</span> <span class="n">password</span><span class="o">);</span>
<span class="o">}</span>
<span class="o">}</span>
<span class="kd">class</span> <span class="nc">ApiUserRegistrationStrategy</span>
<span class="kd">implements</span> <span class="nc">UserRegistrationStrategy</span> <span class="o">{</span>
<span class="nd">@Override</span>
<span class="kd">public</span> <span class="nc">User</span> <span class="nf">register</span><span class="o">()</span> <span class="o">{</span>
<span class="nc">User</span> <span class="n">user</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">User</span><span class="o">(</span><span class="nc">UserRegistry</span><span class="o">.</span><span class="na">getNewUsername</span><span class="o">(),</span> <span class="nc">PasswordGenerator</span><span class="o">.</span><span class="na">generatePassword</span><span class="o">());</span>
<span class="n">put</span><span class="o">(</span><span class="s">"api/user"</span><span class="o">).</span><span class="na">withBody</span><span class="o">(</span><span class="n">user</span><span class="o">.</span><span class="na">toJson</span><span class="o">());</span>
<span class="k">return</span> <span class="n">user</span><span class="o">;</span>
<span class="o">}</span>
<span class="o">}</span>
</code></pre></div></div>
<p>You may not want to invoke “long” registration every time in your tests. But sometimes you’ll need it, for example when you validate actual registration through the web. And vice versa, we want new user creation for test needs to be fast and reliable. That’s why REST registration would be suitable here.</p>
<p>Strategy helps making our test automation framework more flexible and easier in maintenance by using separation of concepts. Again you need to be careful and not implement it in situations when you can do fine without it.</p>
<p>That’s it for Structural Patterns. I’ll make overview of other pattern types in the next articles:</p>
<ul>
<li><a href="https://alexilyenko.github.io/patterns-2/">Design Patterns in Test Automation II</a></li>
<li><a href="https://alexilyenko.github.io/patterns-3/">Design Patterns in Test Automation III</a></li>
</ul>
<p><a href="https://alexilyenko.github.io/"><img src="https://alexilyenko.github.io/assets/images/share_message.png" alt="Feel free to share!" /></a></p>Alex IlyenkoStructural Patterns in Test Automation ArchitectureParallel iOS/tvOS Tests2017-10-30T00:00:00-04:002017-10-30T00:00:00-04:00https://alexilyenko.github.io/ios-parallel-tests<p>It’s essential to run our tests in parallel to perform more of them in a tighter window, get feedback earlier and release faster. This is especially true for user interface tests which tend to be the most time consuming and flaky group. And Apple has heard our prayers! During the last WWDC the company presented one of the most useful features regarding testing, possibility of running concurrent XCTest sessions on multiple iOS/tvOS devices.</p>
<p>From Xcode 9 and later we are able to run not only the same test on the different devices simultaneously, but also different sets of the tests on different devices. It means that basically Apple introduced <strong>test sharding</strong>. Wonder what that is?</p>
<p>Say you have two iPhone X devices and 10 tests. Now you can split your test suite into 2 shards (parts) and run 5 tests per each device. In theory it should decrease run time by factor of <strong><em>n</em></strong>, where <strong><em>n</em></strong> is the number of shards. More about this concept can be found in one of my previous posts on <a href="https://alexilyenko.github.io/android-parallel/">Parallel Android Tests</a>.</p>
<h2 id="parallel-tests-on-simulators">Parallel tests on Simulators</h2>
<p>Let’s start from easier approach of parallel tests - running them on Simulators. The first question we need to ask ourselves is - how many devices we want to run tests on? You’ll be amazed once you know how many concurrent sessions on Simulators were allowed by Apple. According to <code class="language-plaintext highlighter-rouge">xcodebuild</code> logs - it’s <strong>limitless</strong>! Awesome, huh?</p>
<p>The simplest way to run iOS tests on Simulators consists of the following steps:</p>
<ul>
<li>Creating additional <strong><em>n</em></strong> UI test schemes</li>
<li>Splitting existing test suite between created schemes</li>
<li>Creating additional <strong><em>n</em></strong> iOS Simulators (optional, if you want to run on different device models)</li>
<li>Running parallel tests on <strong><em>n</em></strong> simulators differentiating them by names/UDIDs</li>
</ul>
<h3 id="creating-test-targets">Creating test targets</h3>
<p>In Xcode 9 creating UI Test Targets is rather easy. If you already have UI Test Target, all you need to do is to duplicate it <strong><em>n</em></strong> times. If not, check out my <a href="https://alexilyenko.github.io/xcuitest-basics/">XCUITest Essentials</a> post, where I covered the basics of creating UI Test scheme.</p>
<figure class="">
<img src="/assets/images/test_target_duplication.png" alt="Duplicate your test target" /><figcaption>
Duplicate your test target
</figcaption></figure>
<h3 id="splitting-tests">Splitting tests</h3>
<p>After we created additional schemes we have to split our entire test suite between those. To do that we need to open Scheme Settings by navigating to <strong>Product</strong> > <strong>Scheme</strong> > <strong>Manage Schemes</strong>. Now we have to select created schemes one by one and disable/enable needed tests to shard them between targets.</p>
<figure>
<a href="https://alexilyenko.github.io/assets/images/splitting_tests.png"><img src="https://alexilyenko.github.io/assets/images/splitting_tests.png" /></a>
<figcaption>Splitting tests between newly created targets</figcaption>
</figure>
<h3 id="cloning-simulators">Cloning Simulators</h3>
<p>Since we want to run different tests on the same device type in parallel, we need to create additional simulators of the same model and iOS version. For instance, we want to run our tests on 3 devices of type <code class="language-plaintext highlighter-rouge">iPhone X (iOS 11.0)</code>. This would mean that 2 additional (excluding default one) devices should be created in our system. We could do that in <strong>Simulators</strong> menu under <strong>Window</strong> > <strong>Devices and Simulators</strong>. By pressing <strong>+</strong> we can set new device’s model and iOS versions along with paired watchOS device.</p>
<figure>
<a href="https://alexilyenko.github.io/assets/images/cloning_simulators.png"><img src="https://alexilyenko.github.io/assets/images/cloning_simulators.png" /></a>
<figcaption>Cloning of iPhone X Simulator</figcaption>
</figure>
<h2 id="parallel-test-run">Parallel Test Run</h2>
<p>As I mentioned before there were two options for running iOS/tvOS tests in parallel: <strong><em>same test+different devices</em></strong> and <strong><em>different tests+same device</em></strong>.</p>
<h3 id="same-tests-on-the-different-devices">Same tests on the different devices</h3>
<p>To run same tests on different devices we would need to include multiple <code class="language-plaintext highlighter-rouge">destination</code> flags for each device we want to run test scheme on into <code class="language-plaintext highlighter-rouge">xcodebuild test</code> command:</p>
<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code>xcodebuild <span class="se">\</span>
<span class="nt">-scheme</span> SimpleCalculatorUITests <span class="se">\</span>
<span class="nt">-destination</span> <span class="s1">'platform=iOS Simulator,name=iPhone X,OS=11.0'</span> <span class="se">\</span>
<span class="nt">-destination</span> <span class="s1">'platform=iOS Simulator,name=iPhone 8,OS=11.0'</span> <span class="se">\</span>
<span class="nb">test</span>
</code></pre></div></div>
<p>In this case our test scheme <code class="language-plaintext highlighter-rouge">SimpleCalculatorUITests</code> will be executed on <code class="language-plaintext highlighter-rouge">iPhone X</code> and <code class="language-plaintext highlighter-rouge">iPhone 8</code> simultaneously.</p>
<figure class="">
<img src="/assets/images/same.gif" alt="Running the same test target on different iOS devices in parallel" /><figcaption>
Running the same test target on different iOS devices in parallel
</figcaption></figure>
<h3 id="different-tests-on-the-same-device-test-sharding">Different tests on the same device (test sharding)</h3>
<p>That’s where previously created schemes and additional simulators come in handy. To run different test targets on the same device type we would need to invoke <code class="language-plaintext highlighter-rouge">xcodebuild</code> for each of them, setting different simulator for run as <code class="language-plaintext highlighter-rouge">destination</code> value:</p>
<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code>xcodebuild <span class="se">\</span>
<span class="nt">-scheme</span> SimpleCalculatorUITests <span class="se">\</span>
<span class="nt">-destination</span> <span class="s1">'platform=iOS Simulator,name=iPhone X,OS=11.0'</span> <span class="se">\</span>
<span class="nb">test</span> & <span class="se">\</span>
xcodebuild <span class="se">\</span>
<span class="nt">-scheme</span> <span class="s2">"SimpleCalculatorUITests copy"</span> <span class="se">\</span>
<span class="nt">-destination</span> <span class="s1">'platform=iOS Simulator,name=iPhone X 2,OS=11.0'</span> <span class="se">\</span>
<span class="nb">test</span> & <span class="se">\</span>
xcodebuild <span class="se">\</span>
<span class="nt">-scheme</span> <span class="s2">"SimpleCalculatorUITests copy 2"</span> <span class="se">\</span>
<span class="nt">-destination</span> <span class="s1">'platform=iOS Simulator,name=iPhone X 3,OS=11.0'</span> <span class="se">\</span>
<span class="nb">test</span> &
</code></pre></div></div>
<p>In this case three different test schemes will be executed on 3 devices of <code class="language-plaintext highlighter-rouge">iPhone X (iOS 11.0)</code> type.</p>
<figure class="">
<img src="/assets/images/parallel_run_ios.gif" alt="Running different test targets on different iOS devices in parallel" /><figcaption>
Running different test targets on different iOS devices in parallel
</figcaption></figure>
<h2 id="parallel-test-class-run">Parallel test class run</h2>
<p>Similarly to how we executed test targets in parallel, we are able to run test classes (or even methods) in parallel. <code class="language-plaintext highlighter-rouge">-only-testing</code> flag will help us in doing that. All we need to do is to specify the relative path of the test class:</p>
<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code>xcodebuild <span class="se">\</span>
<span class="nt">-scheme</span> SimpleCalculatorUITests <span class="se">\</span>
<span class="nt">-destination</span> <span class="s1">'platform=iOS Simulator,name=iPhone X,OS=11.0'</span> <span class="se">\</span>
<span class="nt">-only-testing</span>:SimpleCalculatorUITests/MinusTest <span class="se">\</span>
<span class="nt">-only-testing</span>:SimpleCalculatorUITests/ResetTest <span class="se">\</span>
<span class="nb">test</span> & <span class="se">\</span>
xcodebuild <span class="se">\</span>
<span class="nt">-scheme</span> SimpleCalculatorUITests <span class="se">\</span>
<span class="nt">-destination</span> <span class="s1">'platform=iOS Simulator,name=iPhone X 2,OS=11.0'</span> <span class="se">\</span>
<span class="nt">-only-testing</span>:SimpleCalculatorUITests/AdditionTest <span class="se">\</span>
<span class="nb">test</span> & <span class="se">\</span>
xcodebuild <span class="se">\</span>
<span class="nt">-scheme</span> SimpleCalculatorUITests <span class="se">\</span>
<span class="nt">-destination</span> <span class="s1">'platform=iOS Simulator,name=iPhone X 3,OS=11.0'</span> <span class="se">\</span>
<span class="nt">-only-testing</span>:SimpleCalculatorUITests/MultiplyTest <span class="se">\</span>
<span class="nb">test</span> &
</code></pre></div></div>
<p>In the example above <code class="language-plaintext highlighter-rouge">MinusTest</code> and <code class="language-plaintext highlighter-rouge">ResetTest</code> classes will be run on <code class="language-plaintext highlighter-rouge">iPhone X</code> device, <code class="language-plaintext highlighter-rouge">AdditionTest</code> will be executed on
<code class="language-plaintext highlighter-rouge">iPhone X 2</code> and <code class="language-plaintext highlighter-rouge">MultiplyTest</code> one on <code class="language-plaintext highlighter-rouge">iPhone X 3</code>. And all of that will be done in parallel!</p>
<h2 id="auto-splitting-of-tests">Auto splitting of tests</h2>
<p>Previous approach is great, but it’s not perfect. It’s really inconvenient to manually specify which tests to run each time we need to do that, and continuous integration systems are not even smart enough to do something like that. So the next improvement I came across was auto splitting of test classes equally between available simulators:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c">#!/usr/bin/env bash</span>
<span class="nv">devices</span><span class="o">=(</span><span class="s2">"platform=iOS Simulator,name=iPhone X,OS=11.0"</span>
<span class="s2">"platform=iOS Simulator,name=iPhone X 2,OS=11.0"</span>
<span class="s2">"platform=iOS Simulator,name=iPhone X 3,OS=11.0"</span><span class="o">)</span>
<span class="nv">test_scheme_name</span><span class="o">=</span><span class="s1">'SimpleCalculatorUITests'</span>
<span class="k">for</span> <span class="o">((</span> <span class="nv">i</span><span class="o">=</span>0<span class="p">;</span> i<<span class="k">${#</span><span class="nv">devices</span><span class="p">[@]</span><span class="k">}</span><span class="p">;</span> i++ <span class="o">))</span><span class="p">;</span>
<span class="k">do
</span>devices[<span class="nv">$i</span><span class="o">]=</span><span class="s2">"xcodebuild
-scheme </span><span class="nv">$test_scheme_name</span><span class="s2">
-destination '"</span><span class="k">${</span><span class="nv">devices</span><span class="p">[</span><span class="nv">$i</span><span class="p">]</span><span class="k">}</span><span class="s2">"'"</span>
<span class="k">done
</span><span class="nv">i</span><span class="o">=</span>0
<span class="nv">path_to_test_classes</span><span class="o">=</span><span class="s1">'SimpleCalculatorUITests/PageObject/Tests'</span>
<span class="k">for </span>entry <span class="k">in</span> <span class="s2">"</span><span class="nv">$path_to_test_classes</span><span class="s2">"</span>/<span class="k">*</span>Test.swift
<span class="k">do
if</span> <span class="o">((</span> i <span class="o">==</span> <span class="k">${#</span><span class="nv">devices</span><span class="p">[@]</span><span class="k">}</span> <span class="o">))</span><span class="p">;</span> <span class="k">then
</span><span class="nv">i</span><span class="o">=</span>0
<span class="k">fi
</span><span class="nv">name</span><span class="o">=</span><span class="s2">"</span><span class="k">${</span><span class="nv">entry</span><span class="p">##*/</span><span class="k">}</span><span class="s2">"</span>
<span class="nv">name</span><span class="o">=</span><span class="s2">"</span><span class="k">${</span><span class="nv">name</span><span class="p">%.*</span><span class="k">}</span><span class="s2">"</span>
devices[<span class="nv">$i</span><span class="o">]=</span><span class="k">${</span><span class="nv">devices</span><span class="p">[</span><span class="nv">$i</span><span class="p">]</span><span class="k">}</span><span class="s2">"
-only-testing:</span><span class="nv">$test_scheme_name</span><span class="s2">/</span><span class="nv">$name</span><span class="s2">"</span>
<span class="o">((</span>i++<span class="o">))</span>
<span class="k">done
</span><span class="nv">cmd</span><span class="o">=</span><span class="s1">''</span>
<span class="k">for</span> <span class="o">((</span> <span class="nv">i</span><span class="o">=</span>0<span class="p">;</span> i<<span class="k">${#</span><span class="nv">devices</span><span class="p">[@]</span><span class="k">}</span><span class="p">;</span> i++ <span class="o">))</span><span class="p">;</span>
<span class="k">do
</span><span class="nv">cmd</span><span class="o">=</span><span class="k">${</span><span class="nv">cmd</span><span class="k">}${</span><span class="nv">devices</span><span class="p">[</span><span class="nv">$i</span><span class="p">]</span><span class="k">}</span><span class="s2">" test & "</span>
<span class="k">done
</span><span class="nb">echo</span> <span class="k">${</span><span class="nv">cmd</span><span class="k">}</span>
<span class="nb">eval</span> <span class="k">${</span><span class="nv">cmd</span><span class="k">}</span>
</code></pre></div></div>
<p><code class="language-plaintext highlighter-rouge">devices</code> is the array which contains names for all simulators we want to run tests on. First, we add <code class="language-plaintext highlighter-rouge">xcodebuild</code> string to each of the array elements. Second, we search for all the test class files under the path <code class="language-plaintext highlighter-rouge">path_to_test_classes</code> by given pattern (in my case it’s <code class="language-plaintext highlighter-rouge">*Test.swift</code>), extracting filename from each found file path string.</p>
<p>Then we iterate through those filenames and split them across the devices we have in <code class="language-plaintext highlighter-rouge">devices</code> one by one. In case we reached the end of array, index would be reset and test splitting would continue from the first device in list. In the end we add <code class="language-plaintext highlighter-rouge">test</code> command to each array element and join them in one <code class="language-plaintext highlighter-rouge">cmd</code> string. And finally, we evaluate string as a shell command.</p>
<p class="notice--info"><strong>Note:</strong> this script should be wrapped into shell script file and put under the project folder.</p>
<h2 id="headless-test-run">Headless test run</h2>
<p>In Xcode 9 Apple’s gone even further by allowing running parallel tests in headless mode. Now we don’t have to start simulators beforehand anymore and <code class="language-plaintext highlighter-rouge">xcodebuild</code> won’t do that implicitly either. In theory it should decrease execution time and save some system resources, particularly video ones.</p>
<iframe width="560" height="315" src="https://www.youtube.com/embed/KbcsmR-I-pk" frameborder="0" allowfullscreen=""></iframe>
<h2 id="parallel-tests-on-real-devices">Parallel tests on Real Devices</h2>
<p>The process of running parallel tests on real iOS/tvOS devices is more or less similar to one on simulators. But instead of specifying <code class="language-plaintext highlighter-rouge">platform</code>, <code class="language-plaintext highlighter-rouge">name</code> and <code class="language-plaintext highlighter-rouge">OS</code> we would need to specify device’s UDID:</p>
<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code>xcodebuild <span class="se">\</span>
<span class="nt">-scheme</span> SimpleCalculatorUITests <span class="se">\</span>
<span class="nt">-destination</span> <span class="s1">'id=${UDID_1}'</span> <span class="se">\</span>
<span class="nb">test</span> & <span class="se">\</span>
xcodebuild <span class="se">\</span>
<span class="nt">-scheme</span> <span class="s2">"SimpleCalculatorUITests copy"</span> <span class="se">\</span>
<span class="nt">-destination</span> <span class="s1">'id=${UDID_2}'</span> <span class="se">\</span>
<span class="nb">test</span> &
</code></pre></div></div>
<h3 id="splitting-tests-between-all-devices">Splitting tests between all Devices</h3>
<p>I have good news for you - we can reuse <a href="https://alexilyenko.github.io/ios-parallel-tests/#auto-splitting-of-tests">auto splitting script</a> for test execution on real devices. The only thing we’d need to change is <code class="language-plaintext highlighter-rouge">devices</code> array creation before executing the script:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c">#!/usr/bin/env bash</span>
<span class="nv">device_type</span><span class="o">=</span><span class="s1">'iPhone'</span>
<span class="nv">devicesString</span><span class="o">=</span><span class="si">$(</span>system_profiler SPUSBDataType |
<span class="nb">grep</span> <span class="nt">-A</span> 11 <span class="nt">-w</span> <span class="s2">"</span><span class="k">${</span><span class="nv">device_type</span><span class="k">}</span><span class="s2">"</span> |
<span class="nb">grep</span> <span class="s2">"Serial Number"</span> |
<span class="nb">awk</span> <span class="s1">'{ print $3 }'</span><span class="si">)</span>
<span class="nv">devices</span><span class="o">=(</span><span class="k">${</span><span class="nv">devicesString</span><span class="p">// / </span><span class="k">}</span><span class="o">)</span>
<span class="k">for</span> <span class="o">((</span> <span class="nv">i</span><span class="o">=</span>0<span class="p">;</span> i<<span class="k">${#</span><span class="nv">devices</span><span class="p">[@]</span><span class="k">}</span><span class="p">;</span> i++ <span class="o">))</span><span class="p">;</span>
<span class="k">do
</span>devices[<span class="nv">$i</span><span class="o">]=</span><span class="s2">"id=</span><span class="k">${</span><span class="nv">devices</span><span class="p">[</span><span class="nv">$i</span><span class="p">]</span><span class="k">}</span><span class="s2">"</span>
<span class="k">done</span>
</code></pre></div></div>
<p>At first we will collect all UDIDs of devices matching <code class="language-plaintext highlighter-rouge">iPhone</code> into one string type with the help of <code class="language-plaintext highlighter-rouge">system_profiler</code> MacOS utility. Then we will split the string into device array. After that we can use previously implemented auto splitting script.</p>
<p>Leveraging this approach we could run our tests on all available (connected) at the moment devices and it should make our test run strategy truly scalable and robust.</p>
<p><a href="https://alexilyenko.github.io/"><img src="https://alexilyenko.github.io/assets/images/share_message.png" alt="Feel free to share!" /></a></p>Alex IlyenkoGetting the most out from your automated tests by running them in parallelAutomating Apple TV Apps2017-10-21T00:00:00-04:002017-10-21T00:00:00-04:00https://alexilyenko.github.io/apple-tv-automated-tests<h2 id="what-tool-to-use">What tool to use?</h2>
<p>In my previous post on <a href="https://alexilyenko.github.io/androidtv-automated-tests/">Automating Android TV Apps</a> we’ve already discussed fast-growing potential of TV application market and I shared couple of insights on why it’s so important to test those. Just to be clear, all of the concepts given there is also applied to Apple TV apps. That’s why if you missed the article you may want to go back and read it first.</p>
<p>Since TV app development is still an young industry, most of the test tools for it is still immature. Actually I couldn’t find any framework for Apple TV apps testing except the one provided by Apple - <strong>XCTest</strong>. If you are aware of any other frameworks please give me heads up in the comments.</p>
<p>Speaking about XCUITest, it appeared not that bad. It’s a native built into Xcode instrument. This means two major advantages - full support and maintenance from Apple and iOS/tvOS/watchOS devs from your team probably have been already familiar with it if they wrote at least one unit test. In case you or your team haven’t heard about it yet I recommend to read my post about <a href="https://alexilyenko.github.io/xcuitest-basics/">XCUITest Testing Essentials</a>.</p>
<h2 id="navigating-through-apple-tv-app">Navigating through Apple TV app</h2>
<p>Any interaction in XCTest is the simulation of user action. Whereas Apple TV unlike iPhone or iPad has not gotten any touchscreen, the only input source would be the Apple remote controller.</p>
<figure class="">
<img src="/assets/images/appletv_remote.jpg" alt="Apple TV Remote Controller" /><figcaption>
Apple TV Remote Controller
</figcaption></figure>
<p>This is the main thing we’d like to consider building automated testing framework for TV app. In tvOS tests Remote controller is represented by <code class="language-plaintext highlighter-rouge">XCUIRemote</code> object. Let’s take a look how we can obtain it:</p>
<div class="language-swift highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">let</span> <span class="nv">remote</span><span class="p">:</span> <span class="kt">XCUIRemote</span> <span class="o">=</span> <span class="kt">XCUIRemote</span><span class="o">.</span><span class="n">shared</span>
</code></pre></div></div>
<p>Now as we know how to get the Remote let’s find out the main commands for interacting with it:</p>
<div class="language-swift highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">remote</span><span class="o">.</span><span class="nf">press</span><span class="p">(</span><span class="o">.</span><span class="n">left</span><span class="p">)</span>
<span class="n">remote</span><span class="o">.</span><span class="nf">press</span><span class="p">(</span><span class="o">.</span><span class="n">right</span><span class="p">)</span>
<span class="n">remote</span><span class="o">.</span><span class="nf">press</span><span class="p">(</span><span class="o">.</span><span class="n">down</span><span class="p">)</span>
<span class="n">remote</span><span class="o">.</span><span class="nf">press</span><span class="p">(</span><span class="o">.</span><span class="n">up</span><span class="p">)</span>
<span class="n">remote</span><span class="o">.</span><span class="nf">press</span><span class="p">(</span><span class="o">.</span><span class="n">select</span><span class="p">)</span>
<span class="n">remote</span><span class="o">.</span><span class="nf">press</span><span class="p">(</span><span class="o">.</span><span class="n">menu</span><span class="p">)</span>
<span class="n">remote</span><span class="o">.</span><span class="nf">press</span><span class="p">(</span><span class="o">.</span><span class="n">playPause</span><span class="p">)</span>
</code></pre></div></div>
<p>Also any button can be pressed with some custom duration. This may be useful for long-presses to rewind video clip or music track:</p>
<div class="language-swift highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">remote</span><span class="o">.</span><span class="nf">press</span><span class="p">(</span><span class="o">.</span><span class="n">right</span><span class="p">,</span> <span class="nv">forDuration</span><span class="p">:</span> <span class="kt">TimeInterval</span><span class="p">(</span><span class="mi">5</span><span class="p">))</span>
</code></pre></div></div>
<p>You may have already figured out that to select element we’d need to move focus onto it. And similarly to Android TV this part might be a little bit tricky in terms of automation. We’d need to build the algorithm for finding path to element depending on the container (i.e. row, column etc.).</p>
<figure class="">
<img src="/assets/images/navigation_tvos.gif" alt="Navigating through tvOS app" /><figcaption>
Navigating through tvOS app
</figcaption></figure>
<p>First let’s take a look at the several helper methods to navigate through your app like in the figure above:</p>
<ul>
<li><code class="language-plaintext highlighter-rouge">focused</code> returns the element which is focused at the moment.
<div class="language-swift highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">var</span> <span class="nv">focused</span><span class="p">:</span> <span class="kt">XCUIElement</span> <span class="p">{</span>
<span class="k">return</span> <span class="n">app</span><span class="o">.</span><span class="nf">descendants</span><span class="p">(</span><span class="nv">matching</span><span class="p">:</span> <span class="o">.</span><span class="n">any</span><span class="p">)</span>
<span class="o">.</span><span class="nf">element</span><span class="p">(</span><span class="nv">matching</span><span class="p">:</span> <span class="kt">NSPredicate</span><span class="p">(</span><span class="nv">format</span><span class="p">:</span> <span class="s">"hasFocus == true"</span><span class="p">)</span>
<span class="p">}</span>
</code></pre></div> </div>
</li>
<li><code class="language-plaintext highlighter-rouge">hasFocus</code> - one of the properties of <code class="language-plaintext highlighter-rouge">XCUIElement</code> object. Determines if the particular element is focused.
<div class="language-swift highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">let</span> <span class="nv">isFocused</span><span class="p">:</span> <span class="kt">Bool</span> <span class="o">=</span> <span class="n">element</span><span class="o">.</span><span class="n">hasFocus</span>
</code></pre></div> </div>
</li>
<li><code class="language-plaintext highlighter-rouge">exists</code> - another useful <code class="language-plaintext highlighter-rouge">XCUIElement</code>’s property. It indicates if element is visible and present on the screen.
<div class="language-swift highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">let</span> <span class="nv">isVisible</span><span class="p">:</span> <span class="kt">Bool</span> <span class="o">=</span> <span class="n">element</span><span class="o">.</span><span class="n">exists</span>
</code></pre></div> </div>
</li>
</ul>
<p>The next I’ll go through couple of the most common containers in tvOS apps and will show how to implement path search strategy for each of them.</p>
<h2 id="strategies-for-finding-elements-in-apple-tv-app">Strategies for finding elements in Apple TV app</h2>
<h3 id="finding-element-in-the-column">Finding element in the column</h3>
<p>We’ll start from the columns, which are usually represented by shelf names and menu titles. It’s the top priority approach we have to implement because any initial step in our tests (e.g. login or choosing the menu) will fail without it.</p>
<figure class="">
<img src="/assets/images/column_tvos.gif" alt="Finding element in the column in Apple TV app" /><figcaption>
Finding element in the column
</figcaption></figure>
<p>To find and focus element in any column we’d need to consider couple of things:</p>
<ol>
<li>Movement direction is either up or down</li>
<li>Element may be located in any direction (above or below) from the currently focused one</li>
<li>Element can be already focused which means all we have to do is to press select button</li>
<li>We need to know when the movement direction should be changed</li>
<li>At some point we’ll have to figure out when to break cycle if element was not found</li>
</ol>
<p>Keeping all those thoughts in mind we can build trivial approach for finding element in the columns:</p>
<div class="language-swift highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">func</span> <span class="nf">findInColumn</span><span class="p">(</span><span class="n">_</span> <span class="nv">element</span><span class="p">:</span> <span class="kt">XCUIElement</span><span class="p">)</span> <span class="p">{</span>
<span class="k">var</span> <span class="nv">isEndReached</span> <span class="o">=</span> <span class="kc">false</span>
<span class="k">while</span> <span class="o">!</span><span class="n">element</span><span class="o">.</span><span class="n">exists</span> <span class="o">||</span> <span class="o">!</span><span class="n">element</span><span class="o">.</span><span class="n">hasFocus</span> <span class="p">{</span>
<span class="k">let</span> <span class="nv">previous</span><span class="p">:</span> <span class="kt">String</span> <span class="o">=</span> <span class="n">focused</span><span class="o">.</span><span class="n">details</span>
<span class="nf">moveFocus</span><span class="p">(</span><span class="n">isEndReached</span><span class="p">)</span>
<span class="k">let</span> <span class="nv">current</span><span class="p">:</span> <span class="kt">String</span> <span class="o">=</span> <span class="n">focused</span><span class="o">.</span><span class="n">details</span>
<span class="k">if</span> <span class="n">previous</span> <span class="o">==</span> <span class="n">current</span> <span class="p">{</span>
<span class="k">if</span> <span class="n">isEndReached</span> <span class="p">{</span>
<span class="kt">XCTFail</span><span class="p">(</span><span class="s">"Element </span><span class="se">\(</span><span class="n">element</span><span class="se">)</span><span class="s"> was not"</span> <span class="o">+</span>
<span class="s">" found in column or could not be focused"</span><span class="p">)</span> <span class="p">}</span>
<span class="p">}</span>
<span class="n">isEndReached</span> <span class="o">=</span> <span class="kc">true</span>
<span class="p">}</span>
<span class="nf">print</span><span class="p">(</span><span class="s">"Element </span><span class="se">\(</span><span class="n">element</span><span class="se">)</span><span class="s"> was found and focused"</span><span class="p">)</span>
<span class="kt">XCUIRemote</span><span class="o">.</span><span class="n">shared</span><span class="o">.</span><span class="nf">press</span><span class="p">(</span><span class="o">.</span><span class="n">select</span><span class="p">)</span>
<span class="p">}</span>
<span class="kd">func</span> <span class="nf">moveFocus</span><span class="p">(</span><span class="n">_</span> <span class="nv">isEndReached</span><span class="p">:</span> <span class="kt">Bool</span><span class="p">)</span> <span class="p">{</span>
<span class="kt">XCUIRemote</span><span class="o">.</span><span class="n">shared</span><span class="o">.</span><span class="nf">press</span><span class="p">(</span><span class="n">isEndReached</span> <span class="p">?</span> <span class="o">.</span><span class="nv">up</span> <span class="p">:</span> <span class="o">.</span><span class="n">down</span><span class="p">)</span>
<span class="p">}</span>
</code></pre></div></div>
<p>The explanation is not hard either. At first we’re moving down through the column elements one by one, checking if the target element is visible and focused. While doing that we also check if end of the column is reached by comparing previously focused element to current one. In case the end is reached we have to change our direction and inspect the upper side of the column. After checking the column for the second time by moving up we will fail the test if the end is reached again since it signals that element was not found.</p>
<h3 id="comparing-xcuielements">Comparing XCUIElements</h3>
<p>Now let’s stop for the moment and take a glance on how elements are compared in the code snippet above. You may have noticed I’m using custom method <code class="language-plaintext highlighter-rouge">details</code> for doing that. The reason for this is the fact that XCUITest does not contain any appropriate mechanism to verify equality of the <code class="language-plaintext highlighter-rouge">XCUIElement</code> objects. Thus I implemented my own extension for that:</p>
<div class="language-swift highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">extension</span> <span class="kt">XCUIElement</span> <span class="p">{</span>
<span class="k">var</span> <span class="nv">details</span><span class="p">:</span> <span class="kt">String</span> <span class="p">{</span>
<span class="k">let</span> <span class="nv">regex</span> <span class="o">=</span> <span class="k">try!</span> <span class="kt">NSRegularExpression</span><span class="p">(</span><span class="nv">pattern</span><span class="p">:</span> <span class="s">"0x</span><span class="se">\\</span><span class="s">S+"</span><span class="p">,</span>
<span class="nv">options</span><span class="p">:</span> <span class="o">.</span><span class="n">caseInsensitive</span><span class="p">)</span>
<span class="k">return</span> <span class="n">regex</span><span class="o">.</span><span class="nf">stringByReplacingMatches</span><span class="p">(</span><span class="nv">in</span><span class="p">:</span> <span class="n">debugDescription</span><span class="p">,</span>
<span class="nv">options</span><span class="p">:</span> <span class="p">[],</span> <span class="nv">range</span><span class="p">:</span> <span class="kt">NSMakeRange</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="n">debugDescription</span><span class="o">.</span><span class="n">count</span><span class="p">),</span>
<span class="nv">withTemplate</span><span class="p">:</span> <span class="s">""</span><span class="p">)</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>If you read my previous post about <a href="https://alexilyenko.github.io/xcuitest-basics/">XCUITest Basics</a>, you already know that each <code class="language-plaintext highlighter-rouge">XCUIElement</code> has <code class="language-plaintext highlighter-rouge">debugDescription</code> variable which represents its detailed debugging information. So basically if two elements had same debug description, they would be the same element in terms of the application.</p>
<figure>
<a href="https://alexilyenko.github.io/assets/images/debugdescription.png"><img src="https://alexilyenko.github.io/assets/images/debugDescription.png" /></a>
<figcaption>debugDescription output for some button</figcaption>
</figure>
<p>But there is one limitation here. Debug description is generated in runtime and most of the contained data elements get ids based on the current system time. They are represented by hexadecimal numbers in the debug information string (e.g. <code class="language-plaintext highlighter-rouge">0x60000018c300</code>). That’s why for successfully comparing of two elements I had to get rid of them using Regex <code class="language-plaintext highlighter-rouge">0x\\S+</code>.</p>
<h3 id="finding-element-in-the-row">Finding element in the row</h3>
<p>Usually most of the media data in Apple TV is stored in so called “shelves”. Since these shelves are no different than usual rows, we’ll need to implement some approach for finding elements in those.</p>
<figure class="">
<img src="/assets/images/row_tvos.gif" alt="Finding element in the row in tvOS app" /><figcaption>
Finding element in the row
</figcaption></figure>
<p>We’ll use the same code as we leveraged for column look-up with one single change <code class="language-plaintext highlighter-rouge">moveFocus</code> function.</p>
<div class="language-swift highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">func</span> <span class="nf">moveFocus</span><span class="p">(</span><span class="n">_</span> <span class="nv">isEndReached</span><span class="p">:</span> <span class="kt">Bool</span><span class="p">)</span> <span class="p">{</span>
<span class="kt">XCUIRemote</span><span class="o">.</span><span class="n">shared</span><span class="o">.</span><span class="nf">press</span><span class="p">(</span><span class="n">isEndReached</span> <span class="p">?</span> <span class="o">.</span><span class="nv">right</span> <span class="p">:</span> <span class="o">.</span><span class="n">left</span><span class="p">)</span>
<span class="p">}</span>
</code></pre></div></div>
<p><code class="language-plaintext highlighter-rouge">.down</code> was replaced by <code class="language-plaintext highlighter-rouge">.right</code> one, and <code class="language-plaintext highlighter-rouge">.up</code> by <code class="language-plaintext highlighter-rouge">.left</code>. This way we are able to inspect left and right side of the given row.</p>
<h3 id="finding-element-in-the-infinite-carousel">Finding element in the infinite carousel</h3>
<p>Infinite carousels are the common containers in tvOS apps. Basically they are the same as rows but without end.</p>
<figure class="">
<img src="/assets/images/infinite.gif" alt="Finding element in the infinite carousel in Apple TV app" /><figcaption>
Finding element in the infinite carousel
</figcaption></figure>
<p>For carousel search we could use the same algorithm we used before, but with minor changes:</p>
<div class="language-swift highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">func</span> <span class="nf">findInCarousel</span><span class="p">(</span><span class="n">_</span> <span class="nv">element</span><span class="p">:</span> <span class="kt">XCUIElement</span><span class="p">)</span> <span class="p">{</span>
<span class="k">let</span> <span class="nv">first</span><span class="p">:</span> <span class="kt">String</span> <span class="o">=</span> <span class="n">focused</span><span class="o">.</span><span class="n">details</span>
<span class="k">while</span> <span class="o">!</span><span class="n">element</span><span class="o">.</span><span class="n">exists</span> <span class="o">||</span> <span class="o">!</span><span class="n">element</span><span class="o">.</span><span class="n">hasFocus</span> <span class="p">{</span>
<span class="kt">XCUIRemote</span><span class="o">.</span><span class="n">shared</span><span class="o">.</span><span class="nf">press</span><span class="p">(</span><span class="o">.</span><span class="n">right</span><span class="p">)</span>
<span class="k">let</span> <span class="nv">current</span><span class="p">:</span> <span class="kt">String</span> <span class="o">=</span> <span class="n">focused</span><span class="o">.</span><span class="n">details</span>
<span class="k">if</span> <span class="n">current</span> <span class="o">==</span> <span class="n">first</span> <span class="p">{</span>
<span class="kt">XCTFail</span><span class="p">(</span><span class="s">"Element </span><span class="se">\(</span><span class="n">element</span><span class="se">)</span><span class="s"> was not"</span> <span class="o">+</span>
<span class="s">" found in the carousel or could not be focused"</span><span class="p">)</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="nf">print</span><span class="p">(</span><span class="s">"Element </span><span class="se">\(</span><span class="n">element</span><span class="se">)</span><span class="s"> was found and focused"</span><span class="p">)</span>
<span class="kt">XCUIRemote</span><span class="o">.</span><span class="n">shared</span><span class="o">.</span><span class="nf">press</span><span class="p">(</span><span class="o">.</span><span class="n">select</span><span class="p">)</span>
<span class="p">}</span>
</code></pre></div></div>
<p>At the beginning we memorize focused element and then while moving and searching for target we compare every next focused one to the first one. Once focused one is equal to first memorized element we’ve made full circle and the test should be failed.</p>
<h3 id="finding-element-in-the-grid">Finding element in the grid</h3>
<p>Grid is the common place for storing collections of elements in order to search and navigate through them.</p>
<iframe width="560" height="315" src="https://www.youtube.com/embed/QhVjys6dZ9o?rel=0" frameborder="0" allowfullscreen=""></iframe>
<p>In my opinion, grid is the most difficult container in terms of automation. There are bunch of things you need to think of while looking up for some element in it</p>
<ol>
<li>If we want to find element in one iteration we need to start search from the very first element in the grid which is top left one</li>
<li>Grid’s row may not be completely visible. Thus if that’s the case for you, you’ll need to completely inspect whole row before moving further</li>
<li>Speaking of optimization, to inspect everything in one iteration we’ll have to move in different directions for each row (i.e. at first to the right, then - to the left, after that - to the right again and so on)</li>
<li>Grids might be asymmetrical (last row is incomplete), and that would require to inspect both very bottom elements (right and left) to make sure there is no transition to the next row possible</li>
</ol>
<p>This scheme may help you to understand algorithm’s logic:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>1 -> 2 -> 3 -> 4 -> 5
|
v
6 <- 7 <- 8 <- 9 <- 10
|
v
11 -> 12 -> 13 -> 14 -> 15
<- <- <- <-
|
v
16 -> 17 -> 18
</code></pre></div></div>
<p>The grid in this scheme is asymmetrical for demonstration purposes. Just after the the bottom right (i.e. 15th one) element is approached (<code class="language-plaintext highlighter-rouge">isEndReached = true</code>), algorithm will change direction and go to the most left bottom element (i.e 11th one) to search for possible incomplete row.</p>
<p>Here is complete implementation of the Grid look-up :</p>
<div class="language-swift highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">func</span> <span class="nf">findInGrid</span><span class="p">(</span><span class="n">_</span> <span class="nv">element</span><span class="p">:</span> <span class="kt">XCUIElement</span><span class="p">)</span> <span class="p">{</span>
<span class="k">var</span> <span class="nv">isMovingRight</span> <span class="o">=</span> <span class="kc">true</span>
<span class="k">var</span> <span class="nv">current</span><span class="p">:</span> <span class="kt">String</span>
<span class="k">var</span> <span class="nv">previous</span><span class="p">:</span> <span class="kt">String</span><span class="p">?</span> <span class="o">=</span> <span class="kc">nil</span>
<span class="k">var</span> <span class="nv">isEndReached</span> <span class="o">=</span> <span class="kc">false</span>
<span class="k">while</span> <span class="o">!</span><span class="n">element</span><span class="o">.</span><span class="n">exists</span> <span class="o">||</span> <span class="o">!</span><span class="n">element</span><span class="o">.</span><span class="n">hasFocus</span> <span class="p">{</span>
<span class="nf">moveFocus</span><span class="p">(</span><span class="n">isMovingRight</span><span class="p">)</span>
<span class="n">current</span> <span class="o">=</span> <span class="n">focused</span><span class="o">.</span><span class="n">details</span>
<span class="k">if</span> <span class="n">current</span> <span class="o">==</span> <span class="n">previous</span> <span class="p">{</span>
<span class="kt">XCUIRemote</span><span class="o">.</span><span class="n">shared</span><span class="o">.</span><span class="nf">press</span><span class="p">(</span><span class="o">.</span><span class="n">down</span><span class="p">)</span>
<span class="n">isMovingRight</span> <span class="o">=</span> <span class="o">!</span><span class="n">isMovingRight</span>
<span class="n">current</span> <span class="o">=</span> <span class="n">focused</span><span class="o">.</span><span class="n">details</span>
<span class="k">if</span> <span class="n">current</span> <span class="o">==</span> <span class="n">previous</span> <span class="p">{</span>
<span class="k">if</span> <span class="n">isEndReached</span> <span class="p">{</span>
<span class="kt">XCTFail</span><span class="p">(</span><span class="s">"Element </span><span class="se">\(</span><span class="n">element</span><span class="se">)</span><span class="s"> was not"</span> <span class="o">+</span>
<span class="s">" found in the grid"</span><span class="p">)</span>
<span class="p">}</span> <span class="k">else</span> <span class="p">{</span>
<span class="n">isEndReached</span> <span class="o">=</span> <span class="kc">true</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="n">previous</span> <span class="o">=</span> <span class="n">current</span>
<span class="p">}</span>
<span class="nf">print</span><span class="p">(</span><span class="s">"Element </span><span class="se">\(</span><span class="n">element</span><span class="se">)</span><span class="s"> was found!"</span><span class="p">)</span>
<span class="kt">XCUIRemote</span><span class="o">.</span><span class="n">shared</span><span class="o">.</span><span class="nf">press</span><span class="p">(</span><span class="o">.</span><span class="n">select</span><span class="p">)</span>
<span class="p">}</span>
<span class="kd">func</span> <span class="nf">moveFocus</span><span class="p">(</span><span class="n">_</span> <span class="nv">isMovingRight</span><span class="p">:</span> <span class="kt">Bool</span><span class="p">)</span> <span class="p">{</span>
<span class="kt">XCUIRemote</span><span class="o">.</span><span class="n">shared</span><span class="o">.</span><span class="nf">press</span><span class="p">(</span><span class="n">isMovingRight</span> <span class="p">?</span> <span class="o">.</span><span class="nv">right</span> <span class="p">:</span> <span class="o">.</span><span class="n">left</span><span class="p">)</span>
<span class="p">}</span>
</code></pre></div></div>
<p>The basic idea is the same as ones from previous strategies. The key difference is that we move focus one level lower after end of the row and only then we change direction. Also to make sure we inspect the last row in case of asymmetrical grid we check left and right sides of the last/pre-last row.</p>
<h3 id="further-optimizations">Further optimizations</h3>
<p>No doubt, all the of the given algorithms could be optimized in one way or another. For example, you may increase the amount of steps if element is not visible to speed up selection movement. Also you could skip the whole row in Grid Search if all of its elements are visible and there is no target element spotted yet.</p>
<p>The next step in terms of optimization I would do, is extracting duplicated code into some method. Usually I create enum <code class="language-plaintext highlighter-rouge">Direction</code> and store such values as <code class="language-plaintext highlighter-rouge">.row</code>, <code class="language-plaintext highlighter-rouge">.column</code> or <code class="language-plaintext highlighter-rouge">.grid</code> there. This helps to determine which direction should be chosen if I reuse the same method <code class="language-plaintext highlighter-rouge">findElement</code> for all strategies.</p>
<p>The further move to increase the code health of your automated framework is the proper object oriented design. It can be achieved by implementing well-known test automation patters, like <a href="https://alexilyenko.github.io/xcuitest-page-object/">Page Object Pattern</a>.</p>
<p>You may already know, that it’s essential to parallel tests to perform more of them in a tighter window, get feedback earlier and release faster. So another thing on the road to successful automation would be simultaneous run of XCUITests on multiple tvOS devices. I will explain how to do that in one of my next posts.</p>
<p>If you have your own thoughts or experiences regarding optimization of Apple TV tests, please share them in the comments.</p>
<p><a href="https://alexilyenko.github.io/"><img src="https://alexilyenko.github.io/assets/images/share_message.png" alt="Feel free to share!" /></a></p>Alex IlyenkoRevealing key points of building successful tvOS test automationPage Object in XCTest UI Tests2017-10-14T00:00:00-04:002017-10-14T00:00:00-04:00https://alexilyenko.github.io/xcuitest-page-object<h2 id="why-page-object-pattern">Why Page Object Pattern</h2>
<p>We already discussed the Page (Screen) Object Pattern and why you should consider using it in development of your test automation architecture in <a href="https://alexilyenko.github.io/uiautomator-page-object/">Page Object for Android UI Tests</a> post. In today’s article we won’t go into deep, instead I’ll shed the light on the key points of using the pattern and share couple of practical examples. But if you’re interested in the details don’t hesitate to check out my original post about it.</p>
<p class="notice--info">The snippets below are written in Swift, but the concept can be applied to any programming language, including Objective-C. All of the examples can be found in <a href="https://github.com/alexilyenko/SimpleIOSCalculator/tree/develop">GiHub Repo</a>, which contains iOS Calculator application and user interface test suite for the app verification.</p>
<p>At first let’s take a look at the hypothetical example of UI test for calculator app:</p>
<div class="language-swift highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">func</span> <span class="nf">testPlus</span><span class="p">()</span> <span class="p">{</span>
<span class="n">app</span><span class="o">.</span><span class="n">buttons</span><span class="p">[</span><span class="s">"8"</span><span class="p">]</span><span class="o">.</span><span class="nf">tap</span><span class="p">()</span>
<span class="n">app</span><span class="o">.</span><span class="n">buttons</span><span class="p">[</span><span class="s">"+"</span><span class="p">]</span><span class="o">.</span><span class="nf">tap</span><span class="p">()</span>
<span class="n">app</span><span class="o">.</span><span class="n">buttons</span><span class="p">[</span><span class="s">"2"</span><span class="p">]</span><span class="o">.</span><span class="nf">tap</span><span class="p">()</span>
<span class="n">app</span><span class="o">.</span><span class="n">buttons</span><span class="p">[</span><span class="s">"="</span><span class="p">]</span><span class="o">.</span><span class="nf">tap</span><span class="p">()</span>
<span class="k">let</span> <span class="nv">result</span> <span class="o">=</span> <span class="kt">Double</span><span class="p">(</span><span class="n">app</span><span class="o">.</span><span class="n">staticTexts</span><span class="o">.</span><span class="n">firstMatch</span><span class="o">.</span><span class="n">label</span><span class="p">)</span>
<span class="kt">XCTAssertEqual</span><span class="p">(</span><span class="mf">10.0</span><span class="p">,</span> <span class="n">result</span><span class="p">,</span>
<span class="s">"Result should be equal to 10"</span><span class="p">)</span>
<span class="p">}</span>
</code></pre></div></div>
<p>This test’s gotten lots of problems:</p>
<ul>
<li>Assuming we have bunch of tests like that and our UI is going to be frequently changed we’ll be constantly fixing our tests to support it</li>
<li>Test is hard to understand. OK, this one is easy, but real life test case might have complicated logic for transitions between views and operating with objects. And dealing with inner UI elements instead of paying attention to domain specific things makes it harder to understand, especially for app developers or non-tech savvy people from management.</li>
<li>Outrageous code duplication. Did you pay attention how we found the elements for interaction? We’ll have to find the same elements and repeat same steps every time which contradicts to one of the main programming principles - reduce code duplication.</li>
<li>App interactions and data are coupled. This one is a big problem since it won’t allow us to implement DDT and we’ll have to run our tests on the predefined set of data, which may be the reason for flaky tests to appear.</li>
</ul>
<p>All of those problems could be solved by leveraging of Page / Screen Object Pattern in our iOS user interface tests. I like to illustrate this pattern with simple diagram containing screen objects of Facebook app:</p>
<figure class="">
<img src="/assets/images/page_object_scheme_xctest.jpeg" alt="Page Object for iOS tests" /><figcaption>
Page Object for iOS tests
</figcaption></figure>
<p>Figure above shows that in our hypothetical test fixture we’d have three Page Objects representing Login, Home and Settings views of the app which are accessible for any test script in our framework.</p>
<h2 id="test-setup">Test Setup</h2>
<p>I like to refer to code reusing as the most valuable technique in development since it helps to design straight-forward and easy-to-maintain solutions.
As you already know we will use it for constructing our Screen Objects but obviously those objects can not be utilized without tests, and tests require clean code too. So let’s go ahead and move every thing that is applicable to every tests to one ancestor class - <code class="language-plaintext highlighter-rouge">BaseTest</code>. Good candidates for that would be <code class="language-plaintext highlighter-rouge">setUp</code> and <code class="language-plaintext highlighter-rouge">tearDown</code> methods.</p>
<div class="language-swift highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">class</span> <span class="kt">BaseTest</span><span class="p">:</span> <span class="kt">XCTestCase</span> <span class="p">{</span>
<span class="kd">private</span> <span class="k">let</span> <span class="nv">app</span> <span class="o">=</span> <span class="kt">XCUIApplication</span><span class="p">()</span>
<span class="k">override</span> <span class="kd">func</span> <span class="nf">setUp</span><span class="p">()</span> <span class="p">{</span>
<span class="n">continueAfterFailure</span> <span class="o">=</span> <span class="kc">false</span>
<span class="n">app</span><span class="o">.</span><span class="nf">launch</span><span class="p">()</span>
<span class="p">}</span>
<span class="k">override</span> <span class="kd">func</span> <span class="nf">tearDown</span><span class="p">()</span> <span class="p">{</span>
<span class="k">let</span> <span class="nv">screenshot</span> <span class="o">=</span> <span class="kt">XCUIScreen</span><span class="o">.</span><span class="n">main</span><span class="o">.</span><span class="nf">screenshot</span><span class="p">()</span>
<span class="k">let</span> <span class="nv">fullScreenshotAttachment</span> <span class="o">=</span> <span class="kt">XCTAttachment</span><span class="p">(</span><span class="nv">screenshot</span><span class="p">:</span> <span class="n">screenshot</span><span class="p">)</span>
<span class="n">fullScreenshotAttachment</span><span class="o">.</span><span class="n">lifetime</span> <span class="o">=</span> <span class="o">.</span><span class="n">deleteOnSuccess</span>
<span class="nf">add</span><span class="p">(</span><span class="n">fullScreenshotAttachment</span><span class="p">)</span>
<span class="n">app</span><span class="o">.</span><span class="nf">terminate</span><span class="p">()</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p class="notice--info"><strong>Note</strong>: everything from the snippet above and even more XCTest stuff is explained in one of my previous posts on <a href="https://alexilyenko.github.io/xcuitest-basics/">XCUITest Essentials</a>. Thus if you’re not comfortable with test tool from Apple yet, you might want to take a look at it.</p>
<p>Next we will extend all of our test classes from <code class="language-plaintext highlighter-rouge">BaseTest</code>.</p>
<h2 id="creating-page-object">Creating Page Object</h2>
<p>The same as we used calculator app for UiAutomator examples, we will use iOS Calculator app for XCTest Screen Object creation training in this post.</p>
<figure class="">
<img src="/assets/images/ios_app.jpg" alt="Page/Screen Object for iOS app" /><figcaption>
Simple iOS Calculator
</figcaption></figure>
<p>And again since our app is pretty simple, as well as Android implementation, iOS one will consist only from one Page Object. Let’s call it <code class="language-plaintext highlighter-rouge">Calculator</code>.</p>
<div class="language-swift highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">class</span> <span class="kt">Calculator</span><span class="p">:</span> <span class="kt">BaseScreen</span> <span class="p">{</span>
<span class="kd">private</span> <span class="kd">lazy</span> <span class="k">var</span> <span class="nv">plusButton</span> <span class="o">=</span> <span class="n">buttons</span><span class="p">[</span><span class="s">"+"</span><span class="p">]</span>
<span class="kd">private</span> <span class="kd">lazy</span> <span class="k">var</span> <span class="nv">multiplyButton</span> <span class="o">=</span> <span class="n">buttons</span><span class="p">[</span><span class="s">"*"</span><span class="p">]</span>
<span class="kd">private</span> <span class="kd">lazy</span> <span class="k">var</span> <span class="nv">equalButton</span> <span class="o">=</span> <span class="n">buttons</span><span class="p">[</span><span class="s">"="</span><span class="p">]</span>
<span class="kd">private</span> <span class="kd">lazy</span> <span class="k">var</span> <span class="nv">resetButton</span> <span class="o">=</span> <span class="n">buttons</span><span class="p">[</span><span class="s">"AC"</span><span class="p">]</span>
<span class="kd">private</span> <span class="kd">lazy</span> <span class="k">var</span> <span class="nv">minusButton</span> <span class="o">=</span> <span class="n">buttons</span><span class="p">[</span><span class="s">"-"</span><span class="p">]</span>
<span class="kd">private</span> <span class="kd">lazy</span> <span class="k">var</span> <span class="nv">screen</span> <span class="o">=</span> <span class="nf">findAll</span><span class="p">(</span><span class="o">.</span><span class="n">staticText</span><span class="p">)</span><span class="o">.</span><span class="n">firstMatch</span>
<span class="kd">private</span> <span class="kd">lazy</span> <span class="k">var</span> <span class="nv">buttons</span> <span class="o">=</span> <span class="nf">findAll</span><span class="p">(</span><span class="o">.</span><span class="n">button</span><span class="p">)</span>
<span class="k">var</span> <span class="nv">numberOnScreen</span><span class="p">:</span> <span class="kt">Double</span><span class="p">?</span> <span class="p">{</span>
<span class="k">return</span> <span class="kt">Double</span><span class="p">(</span><span class="n">screen</span><span class="o">.</span><span class="n">label</span><span class="p">)</span>
<span class="p">}</span>
<span class="kd">func</span> <span class="nf">enter</span><span class="p">(</span><span class="n">_</span> <span class="nv">number</span><span class="p">:</span> <span class="kt">Double</span><span class="p">)</span> <span class="o">-></span> <span class="kt">Calculator</span> <span class="p">{</span>
<span class="nf">inputNumber</span><span class="p">(</span><span class="n">number</span><span class="p">)</span>
<span class="k">return</span> <span class="k">self</span>
<span class="p">}</span>
<span class="kd">func</span> <span class="nf">plus</span><span class="p">(</span><span class="n">_</span> <span class="nv">number</span><span class="p">:</span> <span class="kt">Double</span><span class="p">)</span> <span class="o">-></span> <span class="kt">Calculator</span> <span class="p">{</span>
<span class="k">return</span> <span class="nf">calculate</span><span class="p">(</span><span class="n">number</span><span class="p">,</span> <span class="n">plusButton</span><span class="p">)</span>
<span class="p">}</span>
<span class="kd">func</span> <span class="nf">multiply</span><span class="p">(</span><span class="nv">byNumber</span><span class="p">:</span> <span class="kt">Double</span><span class="p">)</span> <span class="o">-></span> <span class="kt">Calculator</span> <span class="p">{</span>
<span class="k">return</span> <span class="nf">calculate</span><span class="p">(</span><span class="n">byNumber</span><span class="p">,</span> <span class="n">multiplyButton</span><span class="p">)</span>
<span class="p">}</span>
<span class="kd">private</span> <span class="kd">func</span> <span class="nf">inputNumber</span><span class="p">(</span><span class="n">_</span> <span class="nv">number</span><span class="p">:</span> <span class="kt">Double</span><span class="p">)</span> <span class="p">{</span>
<span class="k">for</span> <span class="n">ch</span> <span class="k">in</span> <span class="kt">String</span><span class="p">(</span><span class="n">number</span><span class="p">)</span><span class="o">.</span><span class="n">characters</span> <span class="p">{</span>
<span class="n">buttons</span><span class="p">[</span><span class="kt">String</span><span class="p">(</span><span class="n">ch</span><span class="p">)]</span><span class="o">.</span><span class="nf">tap</span><span class="p">()</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="kd">func</span> <span class="nf">reset</span><span class="p">()</span> <span class="o">-></span> <span class="kt">Calculator</span> <span class="p">{</span>
<span class="n">resetButton</span><span class="o">.</span><span class="nf">tap</span><span class="p">()</span>
<span class="k">return</span> <span class="k">self</span>
<span class="p">}</span>
<span class="kd">func</span> <span class="nf">minus</span><span class="p">(</span><span class="n">_</span> <span class="nv">number</span><span class="p">:</span> <span class="kt">Double</span><span class="p">)</span> <span class="o">-></span> <span class="kt">Calculator</span> <span class="p">{</span>
<span class="k">return</span> <span class="nf">calculate</span><span class="p">(</span><span class="n">number</span><span class="p">,</span> <span class="n">minusButton</span><span class="p">)</span>
<span class="p">}</span>
<span class="kd">private</span> <span class="kd">func</span> <span class="nf">calculate</span><span class="p">(</span><span class="n">_</span> <span class="nv">number</span><span class="p">:</span> <span class="kt">Double</span><span class="p">,</span>
<span class="n">_</span> <span class="nv">operationButton</span><span class="p">:</span> <span class="kt">XCUIElement</span><span class="p">)</span> <span class="o">-></span> <span class="kt">Calculator</span> <span class="p">{</span>
<span class="n">operationButton</span><span class="o">.</span><span class="nf">tap</span><span class="p">()</span>
<span class="nf">inputNumber</span><span class="p">(</span><span class="n">number</span><span class="p">)</span>
<span class="n">equalButton</span><span class="o">.</span><span class="nf">tap</span><span class="p">()</span>
<span class="k">return</span> <span class="k">self</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>Complete implementation can be found <a href="https://github.com/alexilyenko/SimpleIOSCalculator/blob/develop/SimpleCalculatorUITests/PageObject/Screens/Calculator.swift">here</a>.</p>
<p>The gist above illustrates how usual Page Object class looks like. Pay attention to the variables and methods. All of the UI elements which can be found on the screen (page) are put under variables like <code class="language-plaintext highlighter-rouge">plusButton</code> or <code class="language-plaintext highlighter-rouge">screen</code>. This means that in case our Calculator view’s elements locators change, we’ll have to do minimal work by changing variables here. Besides that all of the methods represent some actions which may be done on the page, like <code class="language-plaintext highlighter-rouge">enter</code> or <code class="language-plaintext highlighter-rouge">plus</code>. They will also help us to encapsulate all page specific things (by reducing the code duplication) and help tests to be more readable.</p>
<p>Additionally I’d like to note that <code class="language-plaintext highlighter-rouge">Calculator</code> class conform to some <code class="language-plaintext highlighter-rouge">BaseScreen</code> protocol. In usual situation it would be some abstract entity representing each and every screen of our app and containing their common variables and methods. It’s another good way to reduce repeatable code. Since we’re not building complex framework here, our protocol contains only one method <code class="language-plaintext highlighter-rouge">findAll</code> for finding elements of the given type on the page. Note, how we simulated abstract class by usual Swift protocol and its extension.</p>
<div class="language-swift highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">protocol</span> <span class="kt">BaseScreen</span> <span class="p">{</span>
<span class="p">}</span>
<span class="kd">extension</span> <span class="kt">BaseScreen</span> <span class="p">{</span>
<span class="kd">func</span> <span class="nf">findAll</span><span class="p">(</span><span class="n">_</span> <span class="nv">type</span><span class="p">:</span> <span class="kt">XCUIElement</span><span class="o">.</span><span class="kt">ElementType</span><span class="p">)</span> <span class="o">-></span> <span class="kt">XCUIElementQuery</span> <span class="p">{</span>
<span class="k">return</span> <span class="n">app</span><span class="o">.</span><span class="nf">descendants</span><span class="p">(</span><span class="nv">matching</span><span class="p">:</span> <span class="n">type</span><span class="p">)</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>Mainly it’s a good place for incapsulating all device specific actions (i.e. methods like <code class="language-plaintext highlighter-rouge">findAll</code>, <code class="language-plaintext highlighter-rouge">isVisible</code>, <code class="language-plaintext highlighter-rouge">getText</code> or <code class="language-plaintext highlighter-rouge">dragAndDrop</code>) and storing common locators.</p>
<h3 id="how-to-find-locators">How to find locators</h3>
<p>The next thing I’d like to talk about is locators or selectors. They are used by XCTest to find UI elements on the screen. I already told my readers how to build and use those in <a href="[XCUITest Essentials](https://alexilyenko.github.io/xcuitest-basics/)">XCTest Basics</a> article, but how does one find them in his app?</p>
<p>There is a simple trick for that! The thing is that each <code class="language-plaintext highlighter-rouge">XCUIElement</code> has <code class="language-plaintext highlighter-rouge">debugDescription</code> variable, which shows the whole subtree under the element if it gets printed. Besides that <code class="language-plaintext highlighter-rouge">XCUIApplication</code> extends that class, which means we can print the whole tree of user interface elements to console by invoking that method on the app instance. This can be done by running simple test in your sandbox:</p>
<div class="language-swift highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">func</span> <span class="nf">testPrintTree</span><span class="p">()</span> <span class="p">{</span>
<span class="nf">print</span><span class="p">(</span><span class="n">app</span><span class="o">.</span><span class="n">debugDescription</span><span class="p">)</span>
<span class="p">}</span>
</code></pre></div></div>
<p>The executed command will give us output like that:</p>
<figure>
<a href="https://alexilyenko.github.io/assets/images/xctest_debug_description.png"><img src="https://alexilyenko.github.io/assets/images/xctest_debug_description.png" /></a>
<figcaption>debugDescription output and actual iOS app in comparison</figcaption>
</figure>
<p>If we compare it to actual app one by one, it will become clear how to generate locators. For example we can observe full tree and the depth of all elements on the screen. We are also able to find out their types, accessibility IDs, labels, ancestors and children here.</p>
<h2 id="implementing-tests">Implementing Tests</h2>
<p>The last but not least thing to do is to implement final test using designed Page Objects:</p>
<div class="language-swift highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">func</span> <span class="nf">testPlus</span><span class="p">()</span> <span class="p">{</span>
<span class="k">let</span> <span class="nv">firstNumber</span> <span class="o">=</span> <span class="n">randomNumber</span>
<span class="k">let</span> <span class="nv">secondNumber</span> <span class="o">=</span> <span class="n">randomNumber</span>
<span class="k">let</span> <span class="nv">calculator</span> <span class="o">=</span> <span class="kt">Calculator</span><span class="p">()</span>
<span class="o">.</span><span class="nf">enter</span><span class="p">(</span><span class="n">firstNumber</span><span class="p">)</span>
<span class="o">.</span><span class="nf">plus</span><span class="p">(</span><span class="n">secondNumber</span><span class="p">)</span>
<span class="kt">XCTAssertEqual</span><span class="p">(</span><span class="n">firstNumber</span> <span class="o">+</span> <span class="n">secondNumber</span><span class="p">,</span>
<span class="n">calculator</span><span class="o">.</span><span class="n">numberOnScreen</span><span class="p">)</span>
<span class="p">}</span>
</code></pre></div></div>
<p>Notice, that the test became much more readable compared to previous implementation. Now it finally speaks in domain specific language instead of exposing inner app details like locators and XCTest interactions. Also it got rid of data coupling. Now we can pass any test data (see <code class="language-plaintext highlighter-rouge">randomNumber</code> variables) and design our tests completely independently from it.</p>
<h2 id="best-practices">Best Practices</h2>
<p>The last thing I wanted to share today is best practices of building you framework with Page Object Pattern.</p>
<ul>
<li>
<p><strong>Same elements/widgets</strong>. In case you have couple of screens containing the same logic you may consider extend both of them from one common parent and encapsulate that logic into it. Usually <code class="language-plaintext highlighter-rouge">BaseScreen</code> is a great fit for that if the “thing” is applicable for all screens extending it.</p>
</li>
<li><strong>Chain Methods</strong>. You might have already noticed this technique in the test example but I’d like to draw additional attention to it. Each of the public methods of your Screen Object should return something - either self, other screen object or data, so we could build our tests in the way usual (i.e. manual) test cases are written. For instance:
<div class="language-swift highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="k">let</span> <span class="nv">result</span> <span class="o">=</span> <span class="n">calculator</span>
<span class="o">.</span><span class="nf">enter</span><span class="p">(</span><span class="n">number1</span><span class="p">)</span>
<span class="o">.</span><span class="nf">plus</span><span class="p">(</span><span class="n">number2</span><span class="p">)</span>
<span class="o">.</span><span class="nf">minus</span><span class="p">(</span><span class="n">number3</span><span class="p">)</span>
<span class="o">.</span><span class="n">numberOnScreen</span>
</code></pre></div> </div>
<p>As I said before our simple framework contains only one Screen Object, but in real world it’s not the case. Here is how returning other Screen Objects may come in handy:</p>
<div class="language-swift highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">let</span> <span class="nv">loginScreen</span><span class="p">:</span> <span class="kt">LoginScreen</span> <span class="o">=</span> <span class="nf">openApp</span><span class="p">()</span>
<span class="k">let</span> <span class="nv">home</span><span class="p">:</span> <span class="kt">HomeScreen</span> <span class="o">=</span> <span class="n">loginScreen</span><span class="o">.</span><span class="nf">loginAs</span><span class="p">(</span><span class="n">defaultUser</span><span class="p">)</span>
</code></pre></div> </div>
</li>
<li><strong>Waits</strong>. Our test is pretty simple and its flow is straight-forward. But in general apps have complicated logic with lots of animations. There are many cases when output of the executed actions can not be seen right away. It may be especially true in the moments when transition between screens happens. Thereby it’s generally a good practice to wait for the condition of fully loaded screen to be satisfied in the constructor of that screen object. The condition is usually represented by visibility of the element:
<div class="language-swift highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nf">init</span><span class="p">()</span> <span class="p">{</span>
<span class="nf">waitUntilVisible</span><span class="p">(</span><span class="nv">element</span><span class="p">:</span> <span class="n">someElement</span><span class="p">,</span>
<span class="nv">timeout</span><span class="p">:</span> <span class="n">someTimeout</span><span class="p">)</span>
<span class="p">}</span>
</code></pre></div> </div>
<p>In my opinion, it’s impossible to build fast and reliable automation without explicit waits. I wrote the article about <a href="https://alexilyenko.github.io/xcuitest-waiting/">Waits in XCUITest</a> and you may want to check it out if you want to master this practice.</p>
</li>
<li><strong>Parallel run</strong>. The last advanced technique for today will be test parallelization. It’s not directly related to Page Object pattern but it affects your testing approach in general. It’s essential to execute your tests simultaneously to perform more of them in a tighter window, get feedback earlier and release faster. I wrote about <a href="https://alexilyenko.github.io/ios-parallel-tests/">Parallel Tests with XCUITest</a> in one of my posts.</li>
</ul>
<p><a href="https://alexilyenko.github.io/"><img src="https://alexilyenko.github.io/assets/images/share_message.png" alt="Feel free to share!" /></a></p>Alex IlyenkoAdvanced technique for building iOS/tvOS testing frameworksWaits in XCUITest2017-10-06T00:00:00-04:002017-10-06T00:00:00-04:00https://alexilyenko.github.io/xcuitest-waiting<h2 id="waiting-is-essential">Waiting is Essential</h2>
<p>The time of the static content on the Web has long gone. It was replaced by dynamic websites and asynchronous mobile applications. Implicit waits cannot help us anymore in avoiding flaky tests while automating those assets. I wrote the whole article about <a href="https://alexilyenko.github.io/uiautomator-waiting/">Waiting in Android Functional Tests</a> and I have to say iOS testing is not different. If you want to build fast and reliable automation for your project, you should implement explicit waits in one way or another.</p>
<p>Today we’ll talk about expectations and waiters in XCTest framework, which might be applied in both unit and functional (user interface) tests of your iOS/tvOS/watchOS app.</p>
<p class="notice--info">If you’ve never worked with XCUITest before, I suggest to read my post about <a href="https://alexilyenko.github.io/xcuitest-basics/">XCTest UI Testing Basics</a> first. I introduced main XCTest classes along with main rules of using them in it.</p>
<h2 id="expectations">Expectations</h2>
<p>First of all, we have to figure out what we’re going to wait for. Basically, this is nothing new, we will wait for some conditions to be satisfied or, if we’re talking in XCUITest language, “some <strong>expectations</strong> to be fulfilled”. While releasing Xcode 7, Apple introduced <a href="https://developer.apple.com/documentation/xctest/xctestexpectation"><code class="language-plaintext highlighter-rouge">XCTestExpectation</code></a> class, which represents expected outcome in an asynchronous test. There are several types of default expectations in XCTest library:</p>
<ul>
<li><strong>XCTKVOExpectation</strong> - expectation which is fulfilled when Key Value Observing condition is met.
<div class="language-swift highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">let</span> <span class="nv">element</span> <span class="o">=</span> <span class="kt">XCUIApplication</span><span class="p">()</span><span class="o">.</span>
<span class="k">let</span> <span class="nv">expectation</span> <span class="o">=</span> <span class="kt">XCTKVOExpectation</span><span class="p">(</span><span class="nv">keyPath</span><span class="p">:</span> <span class="s">"exists"</span><span class="p">,</span>
<span class="nv">object</span><span class="p">:</span> <span class="n">xCUIElement</span><span class="p">,</span>
<span class="nv">expectedValue</span><span class="p">:</span> <span class="kc">true</span><span class="p">)</span>
</code></pre></div> </div>
</li>
<li><strong>XCTNSNotificationExpectation</strong> - expectation which is satisfied when notification is received
<div class="language-swift highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">let</span> <span class="nv">expectation</span> <span class="o">=</span> <span class="kt">XCTNSNotificationExpectation</span><span class="p">(</span><span class="nv">name</span><span class="p">:</span>
<span class="kt">Notification</span><span class="o">.</span><span class="kt">Name</span><span class="p">(</span><span class="s">"MyNotification"</span><span class="p">))</span>
</code></pre></div> </div>
</li>
<li><strong>XCTDarwinNotificationExpectation</strong> - the same as the previous one, with the difference of notification type. In this case we will wait for Darwin notification to be received
<div class="language-swift highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">let</span> <span class="nv">expectation</span> <span class="o">=</span> <span class="kt">XCTDarwinNotificationExpectation</span><span class="p">(</span>
<span class="nv">notificationName</span><span class="p">:</span> <span class="s">"DarwinNotificationName"</span><span class="p">)</span>
</code></pre></div> </div>
</li>
<li><strong>XCTNSPredicateExpectation</strong> - in my opinion the main expectation in functional testing. Basically I think we can not build robust UI automation without this type of expectations. It is fulfilled when given <code class="language-plaintext highlighter-rouge">NSPredicate</code> is satisfied.
<div class="language-swift highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">let</span> <span class="nv">expectation</span> <span class="o">=</span> <span class="kt">XCTNSPredicateExpectation</span><span class="p">(</span><span class="nv">predicate</span><span class="p">:</span>
<span class="kt">NSPredicate</span><span class="p">(</span><span class="nv">format</span><span class="p">:</span> <span class="s">"exists == true"</span><span class="p">),</span>
<span class="nv">object</span><span class="p">:</span> <span class="n">xCUIElement</span><span class="p">)</span>
</code></pre></div> </div>
</li>
</ul>
<p>Let’s talk a bit about <code class="language-plaintext highlighter-rouge">NSPredicate</code>s.</p>
<h3 id="nspredicates">NSPredicates</h3>
<p>I found <code class="language-plaintext highlighter-rouge">NSPredicate</code> the most convenient way for specifying criteria of waiting for <code class="language-plaintext highlighter-rouge">XCUIElement</code> or its attributes in XCUITest framework. With its help we are able to build advanced locators and combine them with each others. For example:</p>
<div class="language-swift highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">let</span> <span class="nv">orNSPredicate</span> <span class="o">=</span> <span class="kt">NSPredicate</span><span class="p">(</span><span class="nv">format</span><span class="p">:</span>
<span class="s">"label CONTAINS 'something' OR name MATCHES 'someRegex'"</span><span class="p">)</span>
<span class="k">let</span> <span class="nv">andNSPredicate</span> <span class="o">=</span> <span class="kt">NSPredicate</span><span class="p">(</span><span class="nv">format</span><span class="p">:</span>
<span class="s">"name BEGINSWITH 'prefix' AND NOT name ENDSWITH 'suffix'"</span><span class="p">)</span>
<span class="k">let</span> <span class="nv">count</span> <span class="o">=</span> <span class="kt">NSPredicate</span><span class="p">(</span><span class="nv">format</span><span class="p">:</span> <span class="s">"self.count = 2"</span><span class="p">)</span>
</code></pre></div></div>
<p>In addition to this, <code class="language-plaintext highlighter-rouge">NSPredicate</code> may be also used in searching for elements, which might be really handy:</p>
<div class="language-swift highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">let</span> <span class="nv">predicate</span> <span class="o">=</span> <span class="kt">NSPredicate</span><span class="p">(</span><span class="nv">format</span><span class="p">:</span> <span class="s">"value CONTAINS 'word'"</span><span class="p">)</span>
<span class="k">let</span> <span class="nv">field</span> <span class="o">=</span> <span class="n">app</span><span class="o">.</span><span class="n">textFields</span><span class="o">.</span><span class="nf">matching</span><span class="p">(</span><span class="n">predicate</span><span class="p">)</span>
</code></pre></div></div>
<h2 id="how-to-wait-in-xctest">How to Wait in XCTest</h2>
<p>There are couple of waiting options built into XCUITest framework.</p>
<h3 id="xctestcase-wait">XCTestCase Wait</h3>
<p>As you may already know, to implement any test in XCTest we have to extend basic <code class="language-plaintext highlighter-rouge">XCTestCase</code> class with our test class. The thing you might not know is this class has already defined method <code class="language-plaintext highlighter-rouge">wait</code>. This means any test in our framework can use it to wait for expectations. It accepts array of expectation objects and synchronously waits for each of them in the given order for given amount of time.</p>
<div class="language-swift highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nf">wait</span><span class="p">(</span><span class="nv">for</span><span class="p">:</span> <span class="p">[</span><span class="n">expectation1</span><span class="p">,</span> <span class="n">expectation2</span><span class="p">],</span>
<span class="nv">timeout</span><span class="p">:</span> <span class="kt">TimeInterval</span><span class="p">(</span><span class="n">timeoutValue</span><span class="p">))</span>
</code></pre></div></div>
<p>I know what you may want to ask. What if we want extract waits to separate util class, which is considered a good practice? If we do that we won’t be able to use <code class="language-plaintext highlighter-rouge">wait</code> method of <code class="language-plaintext highlighter-rouge">XCTestCase</code> class assuming our util class won’t extend it. That’s right, but of course there is a solution out there. Its name is <code class="language-plaintext highlighter-rouge">XCTWaiter</code>.</p>
<h3 id="xctwaiter">XCTWaiter</h3>
<p>This class was introduced by Apple along with Xcode 8 release, which means if you’re using somewhat the latest version, you probably have access to it.</p>
<div class="language-swift highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">_</span> <span class="o">=</span> <span class="kt">XCTWaiter</span><span class="o">.</span><span class="nf">wait</span><span class="p">(</span><span class="nv">for</span><span class="p">:</span> <span class="p">[</span><span class="n">expectation1</span><span class="p">,</span> <span class="n">expectation2</span><span class="p">],</span>
<span class="nv">timeout</span><span class="p">:</span> <span class="kt">TimeInterval</span><span class="p">(</span><span class="n">timeoutValue</span><span class="p">))</span>
</code></pre></div></div>
<p>From the first glance one can say that this method is doing exactly the same as test case’s <code class="language-plaintext highlighter-rouge">wait</code>. But there are hidden advantages in using <code class="language-plaintext highlighter-rouge">XCTWaiter</code>. If you take a closer look at the snippets above, you may notice that the second example returns something. It reveals the object of <code class="language-plaintext highlighter-rouge">XCTWaiter.Result</code> class which represents the result of expectations supposed to be fulfilled. There are four types of result which can be returned by waiter:</p>
<ul>
<li><code class="language-plaintext highlighter-rouge">.completed</code> means condition was satisfied</li>
<li><code class="language-plaintext highlighter-rouge">.timedOut</code> - result which says expectation was not fulfilled during given timeout</li>
<li><code class="language-plaintext highlighter-rouge">.incorrectOrder</code> represents situation when expectations were fulfilled in the order not equal to the given one</li>
<li><code class="language-plaintext highlighter-rouge">.invertedFulfillment</code> is a result that means inverted expectation was fulfilled</li>
<li><code class="language-plaintext highlighter-rouge">.interrupted</code> is saying waiting was interrupted prior to its expectations being fulfilled or timed out</li>
</ul>
<p>The next thing deserving attention is <code class="language-plaintext highlighter-rouge">XCTWaiter</code> won’t fail your test even if expectation was not fulfilled. Now it’s completely our responsibility as developers of the tests. If you think about this, it’s a great improvement. We are now completely in control of when and how to fail our tests. Now it’s possible to wait even for optional conditions without any risks of fails.</p>
<div class="language-swift highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">func</span> <span class="nf">waitForExpectation</span><span class="p">(</span><span class="nv">expectation</span><span class="p">:</span><span class="kt">XCTestExpectation</span><span class="p">,</span>
<span class="nv">time</span><span class="p">:</span> <span class="kt">Double</span><span class="p">,</span>
<span class="nv">safe</span><span class="p">:</span> <span class="kt">Bool</span> <span class="o">=</span> <span class="kc">false</span><span class="p">)</span> <span class="p">{</span>
<span class="k">let</span> <span class="nv">result</span><span class="p">:</span> <span class="kt">XCTWaiter</span><span class="o">.</span><span class="kt">Result</span> <span class="o">=</span>
<span class="kt">XCTWaiter</span><span class="p">()</span><span class="o">.</span><span class="nf">wait</span><span class="p">(</span><span class="nv">for</span><span class="p">:</span> <span class="p">[</span><span class="n">expectation</span><span class="p">],</span>
<span class="nv">timeout</span><span class="p">:</span> <span class="n">time</span><span class="p">)</span>
<span class="k">if</span> <span class="o">!</span><span class="n">safe</span> <span class="o">&&</span> <span class="n">result</span> <span class="o">!=</span> <span class="o">.</span><span class="n">completed</span> <span class="p">{</span>
<span class="c1">// if expectation is strict and was not fulfilled</span>
<span class="kt">XCTFail</span><span class="p">(</span><span class="s">"Condition was not satisfied during </span><span class="se">\(</span><span class="n">time</span><span class="se">)</span><span class="s"> seconds"</span><span class="p">)</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<h3 id="custom-expectations">Custom expectations</h3>
<p>The last thing, I wanted to cover today, is creating your custom expectations and waiting for their fulfillment. There might be situations out there when you want to wait for something, but none of the default expectations is suitable for that. There is a technique for this too!</p>
<p>For example, we need to wait for our asynchronous method <code class="language-plaintext highlighter-rouge">executeSomething</code> to be completed. All we need to do is to invoke <code class="language-plaintext highlighter-rouge">fulfill</code> method of <code class="language-plaintext highlighter-rouge">XCTestExpectation</code> object in the <code class="language-plaintext highlighter-rouge">completion</code> handler block like this:</p>
<div class="language-swift highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">let</span> <span class="nv">expectation</span> <span class="o">=</span> <span class="kt">XCTestExpectation</span><span class="p">(</span><span class="nv">description</span><span class="p">:</span>
<span class="s">"doSomething() is finished"</span><span class="p">)</span>
<span class="nf">executeSomething</span><span class="p">(</span><span class="nv">completion</span><span class="p">:</span> <span class="p">{</span> <span class="n">_</span> <span class="k">in</span>
<span class="nf">doSomething</span><span class="p">()</span>
<span class="n">expectation</span><span class="o">.</span><span class="nf">fulfill</span><span class="p">()</span>
<span class="p">},</span> <span class="nv">failed</span><span class="p">:</span> <span class="p">{</span> <span class="n">_</span> <span class="k">in</span>
<span class="nf">print</span><span class="p">(</span><span class="s">"Something went wrong!"</span><span class="p">)</span>
<span class="p">})</span>
<span class="n">_</span> <span class="o">=</span> <span class="kt">XCTWaiter</span><span class="p">()</span><span class="o">.</span><span class="nf">wait</span><span class="p">(</span><span class="nv">for</span><span class="p">:</span> <span class="p">[</span><span class="n">expectation</span><span class="p">],</span>
<span class="nv">timeout</span><span class="p">:</span> <span class="kt">TimeInterval</span><span class="p">(</span><span class="n">timeoutValue</span><span class="p">))</span>
</code></pre></div></div>
<p>These are the main concepts of waiting in XCUITest. I will explain how they might be used in building successful iOS automation in the next series of posts so stay tuned!
<a href="https://alexilyenko.github.io/"><img src="https://alexilyenko.github.io/assets/images/share_message.png" alt="Feel free to share!" /></a></p>Alex IlyenkoBuilding reliable automation for iOS/tvOS platforms with waits and expectationsXCTest UI Testing Basics2017-09-30T00:00:00-04:002017-09-30T00:00:00-04:00https://alexilyenko.github.io/xcuitest-basics<h2 id="why-xctest">Why XCTest?</h2>
<p>As you may have heard during “Hey Siri” event in 2015 Apple introduced its own interpretation of testing framework, called XCTest (or XCUITest if we’re talking about UI testing). I was surprised to find out that even though it has been already 2 years from its release major part of the Test developers are still being skeptical about this framework.</p>
<p>From my perspective this instrument has gotten everything you need for writing automated tests against iOS/tvOS platforms and I wouldn’t go with anything else unless I need cross-platform (iOS and <a href="https://alexilyenko.github.io/tags/#android">Android</a>) support. And even then I would consider separation of the frameworks. But that’s another topic for discussion.</p>
<p>Here is why I would suggest XCTest over other tools for UI testing of iOS and tvOS applications:</p>
<ul>
<li><strong>No additional components required to be installed</strong>. Starting from Xcode version 7 Apple ships XCTest prebuilt into its development kit and all you need to do is just activate it. I will show how to do that in further reading.</li>
<li><strong>Swift/Objective-C native language support</strong>. If you think of it, it’s pretty big advantage. Since there is no learning curve or language barrier iOS developers and Automation engineers could support/review/extend each other’s test scripts and collaborate on creating new ones.</li>
<li><strong>Easy CI integration</strong>. Xcode allows to execute tests via command line and this is well used by all major CI systems, like Jenkins, TeamCity, Travis or Bamboo. All of them could be easily set up to run iOS functional tests and collect their results.</li>
<li><strong>Speed</strong>. There is no secret that all of the iOS UI test frameworks available on the market are based on XCUITest infrastructure (well, sometimes it may be its ancestor - UI Automation). Obviously something on top would work slower than its lower layer by itself. Finding elements, extracting their attributes, typing text etc - everything is a bit faster in pure XCTest.</li>
<li><strong>Test Recorder</strong>. I don’t use this one but it might be beneficial for beginners to leverage Recorder. One can write actual test steps agains his app and after that see their implementation as an UI test. Yes, it won’t be perfect but with minor changes we could use it in our framework. That said it’s great for learning purposes.</li>
<li><strong>tvOS support</strong>. For the moment there are no reliable frameworks out there which can easily support testing on Apple TV devices. Thus if you want to verify tvOS app, XCUITest is the only way to go. One of my next posts will be completely dedicated to Apple TV testing.</li>
</ul>
<h2 id="how-to-get-started-writing-tests-with-xctest">How to get started writing tests with XCTest</h2>
<p>It’s pretty simple. All we have to do is to add extra target to our Xcode project.</p>
<figure class="">
<img src="/assets/images/add_target_xctest.png" alt="how to configure XCUITest" /><figcaption>
Select Xcode project file and add new target to your project
</figcaption></figure>
<figure class="">
<img src="/assets/images/ios_ui_testing_bundle.png" alt="iOS UI testing setup" /><figcaption>
Find iOS UI Testing Bundle among Target templates
</figcaption></figure>
<figure class="">
<img src="/assets/images/choosing_language_xctest.png" alt="Configure Swift for XCTest UI tests" /><figcaption>
Choose programming language you want to use in your UI tests
</figcaption></figure>
<p>After you went through all the steps above you should see new target was added to your project along with some autogenerated sources. They are not very useful but may give us an idea how to start our app before the test and close it afterwards.</p>
<figure class="">
<img src="/assets/images/result_target_xctest.png" alt="iOS UI Testing target was added to Xcode project" /><figcaption>
New iOS UI Testing target was added to Xcode project
</figcaption></figure>
<h2 id="introduction-to-xcuitest-api">Introduction to XCUITest API</h2>
<p>XCTest is rather light-weight library, all of its classes are conveniently located in one file - <code class="language-plaintext highlighter-rouge">XCTest</code>. Here are the main ones of them:</p>
<ul>
<li><strong><a href="https://developer.apple.com/documentation/xctest/xcuielementquery">XCUIElementQuery</a></strong> - class for querying app’s UI hierarchy to find UI element. For simplicity we could treat element query as an array of all elements, which match to the given criteria. <code class="language-plaintext highlighter-rouge">XCUIElementQuery</code> can be chained with other queries.
<div class="language-swift highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">let</span> <span class="nv">queryForButton</span><span class="p">:</span> <span class="kt">XCUIElementQuery</span> <span class="o">=</span> <span class="n">app</span><span class="o">.</span><span class="n">buttons</span><span class="o">.</span><span class="nf">descendants</span><span class="p">(</span><span class="nv">matching</span><span class="p">:</span> <span class="o">.</span><span class="n">any</span><span class="p">)</span>
</code></pre></div> </div>
</li>
<li><strong><a href="https://developer.apple.com/documentation/xctest/xcuielement">XCUIElement</a></strong> - one of the main classes in XCTest library. It represents UI element in iOS app user interface hierarchy. Lots of stuff can be done to it (we can click on it, read and verify its attribute values, swipe it, drag-and-drop it, input text into it etc).
<div class="language-swift highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">let</span> <span class="nv">firstButton</span><span class="p">:</span> <span class="kt">XCUIElement</span> <span class="o">=</span> <span class="n">app</span><span class="o">.</span><span class="n">buttons</span><span class="o">.</span><span class="n">firstMatch</span>
<span class="n">firstButton</span><span class="o">.</span><span class="nf">click</span><span class="p">()</span>
<span class="k">let</span> <span class="nv">imageWithId</span><span class="p">:</span> <span class="kt">XCUIElement</span> <span class="o">=</span> <span class="n">app</span><span class="o">.</span><span class="n">images</span><span class="p">[</span><span class="s">"identifier"</span><span class="p">]</span>
<span class="n">image</span><span class="o">.</span><span class="nf">swipeUp</span><span class="p">()</span>
</code></pre></div> </div>
</li>
<li><strong><a href="https://developer.apple.com/documentation/xctest/xcuielement.type">XCUIElement.Type</a></strong> - enum for types of the UI elements which can be located. For instance <code class="language-plaintext highlighter-rouge">.image</code>, <code class="language-plaintext highlighter-rouge">.button</code>, <code class="language-plaintext highlighter-rouge">.alert</code>, <code class="language-plaintext highlighter-rouge">.any</code> etc. Types are often used with <code class="language-plaintext highlighter-rouge">XCUIElementQuery</code> to build query for finding UI element.
<div class="language-swift highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">let</span> <span class="nv">cellType</span><span class="p">:</span> <span class="kt">XCUIElement</span><span class="o">.</span><span class="p">`</span><span class="nv">Type</span><span class="p">`</span> <span class="o">=</span> <span class="o">.</span><span class="n">cell</span>
<span class="k">let</span> <span class="nv">queryForImageType</span> <span class="o">=</span> <span class="n">app</span><span class="o">.</span><span class="nf">descendants</span><span class="p">(</span><span class="nv">matching</span><span class="p">:</span> <span class="o">.</span><span class="n">image</span><span class="p">)</span>
</code></pre></div> </div>
</li>
<li><strong><a href="https://developer.apple.com/documentation/xctest/xcuiapplication">XCUIApplication</a></strong> - one of most important classes for building successful UI test framework. It represents iOS application and basically is the only way to interact with it. It will allow us to start and terminate the app, locate elements inside it and verify their attributes. Since <code class="language-plaintext highlighter-rouge">XCUIApplication</code> extends <code class="language-plaintext highlighter-rouge">XCUIElement</code> class, you may consider it as a root element in app hierarchy.
<div class="language-swift highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">let</span> <span class="nv">app</span> <span class="o">=</span> <span class="kt">XCUIApplication</span><span class="p">()</span>
<span class="n">app</span><span class="o">.</span><span class="nf">launch</span><span class="p">()</span>
</code></pre></div> </div>
</li>
<li><strong><a href="https://developer.apple.com/documentation/xctest/xcuiscreen">XCUIScreen</a></strong> - represents physical screen(s) of the iOS/tvOS/macOS device. Its main purpose is snapping screenshots in test runtime.</li>
<li><strong><a href="https://developer.apple.com/documentation/xctest/xcuiscreenshot">XCUIScreenshot</a></strong> - a captured image of a device screen or its part.
<div class="language-swift highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// Take a screenshot of an app's first button.</span>
<span class="k">let</span> <span class="nv">windowScreenshot</span> <span class="o">=</span> <span class="n">app</span><span class="o">.</span><span class="n">button</span><span class="o">.</span><span class="n">firstMatch</span><span class="o">.</span><span class="nf">screenshot</span><span class="p">()</span>
<span class="n">app</span><span class="o">.</span><span class="nf">terminate</span><span class="p">()</span>
<span class="c1">// Take a screenshot of the screen after app was terminated</span>
<span class="k">let</span> <span class="nv">mainScreenScreenshot</span> <span class="o">=</span> <span class="kt">XCUIScreen</span><span class="o">.</span><span class="n">main</span><span class="o">.</span><span class="nf">screenshot</span><span class="p">()</span>
</code></pre></div> </div>
</li>
<li><strong><a href="https://developer.apple.com/documentation/xctest/xcuidevice">XCUIDevice</a></strong> - as you can guess this class represents device which tests are being run on. It’s not frequently used in testing, but you can do some interesting stuff with it. For example, it can help you emulate hardware buttons, check screen orientation and interact with Siri, which was added in the latest release.
<div class="language-swift highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">let</span> <span class="nv">device</span> <span class="o">=</span> <span class="kt">XCUIDevice</span><span class="o">.</span><span class="n">shared</span>
<span class="n">device</span>
<span class="o">.</span><span class="n">siriService</span>
<span class="o">.</span><span class="nf">activate</span><span class="p">(</span><span class="nv">voiceRecognitionText</span><span class="p">:</span> <span class="s">"Hey Siri! How are you?"</span><span class="p">)</span>
</code></pre></div> </div>
</li>
<li><strong><a href="https://developer.apple.com/documentation/xctest/xcuiremote">XCUIRemote</a></strong> - this class simulates interactions done via physical remote controller (e.g. on <a href="https://alexilyenko.github.io/apple-tv-automated-tests/">Apple TV devices</a>).
<div class="language-swift highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">let</span> <span class="nv">remote</span> <span class="o">=</span> <span class="kt">XCUIRemote</span><span class="o">.</span><span class="n">shared</span>
<span class="n">remote</span><span class="o">.</span><span class="nf">press</span><span class="p">(</span><span class="o">.</span><span class="n">playPause</span><span class="p">)</span>
</code></pre></div> </div>
</li>
</ul>
<p>The classes above are the most used by majority of developers in their test frameworks. But of course these are not all of them. Full XCTest API for user interface tests can be found in <a href="https://developer.apple.com/documentation/xctest/user_interface_tests">official documentation</a>.</p>
<h2 id="xctest-ui-test-example">XCTest UI test example</h2>
<p>Assuming we’ve got familiar with main classes in XCUITest library, let’s go ahead and create our first ever test script.</p>
<div class="language-swift highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">class</span> <span class="kt">SimpleTest</span><span class="p">:</span> <span class="kt">XCTestCase</span> <span class="p">{</span>
<span class="c1">// Creating app instance</span>
<span class="k">let</span> <span class="nv">app</span> <span class="o">=</span> <span class="kt">XCUIApplication</span><span class="p">()</span>
<span class="k">override</span> <span class="kd">func</span> <span class="nf">setUp</span><span class="p">()</span> <span class="p">{</span>
<span class="c1">// Setting continue after failure strategy</span>
<span class="n">continueAfterFailure</span> <span class="o">=</span> <span class="kc">false</span>
<span class="c1">// Launching the app</span>
<span class="n">app</span><span class="o">.</span><span class="nf">launch</span><span class="p">()</span>
<span class="p">}</span>
<span class="kd">func</span> <span class="nf">testForm</span><span class="p">()</span> <span class="p">{</span>
<span class="c1">// Finding text field</span>
<span class="k">let</span> <span class="nv">editText</span> <span class="o">=</span> <span class="n">app</span><span class="o">.</span><span class="n">textFields</span><span class="p">[</span><span class="s">"edit_text"</span><span class="p">]</span>
<span class="c1">// Sending text to field</span>
<span class="n">editText</span><span class="o">.</span><span class="nf">typeText</span><span class="p">(</span><span class="s">"123456"</span><span class="p">)</span>
<span class="k">let</span> <span class="nv">submitButton</span> <span class="o">=</span> <span class="n">app</span><span class="o">.</span><span class="n">buttons</span><span class="p">[</span><span class="s">"submit"</span><span class="p">]</span>
<span class="c1">// Tapping on button</span>
<span class="n">submitButton</span><span class="o">.</span><span class="nf">tap</span><span class="p">()</span>
<span class="k">let</span> <span class="nv">alert</span> <span class="o">=</span> <span class="n">app</span><span class="o">.</span><span class="n">alerts</span><span class="o">.</span><span class="n">firstMatch</span>
<span class="c1">// Waiting for alert to appear</span>
<span class="n">_</span> <span class="o">=</span> <span class="n">alert</span><span class="o">.</span><span class="nf">waitForExistence</span><span class="p">(</span><span class="nv">timeout</span><span class="p">:</span> <span class="kt">TimeInterval</span><span class="p">(</span><span class="mi">10</span><span class="p">))</span>
<span class="kt">XCTAssertEqual</span><span class="p">(</span><span class="n">alert</span><span class="o">.</span><span class="n">label</span><span class="p">,</span> <span class="s">"Success!"</span><span class="p">)</span>
<span class="p">}</span>
<span class="k">override</span> <span class="kd">func</span> <span class="nf">tearDown</span><span class="p">()</span> <span class="p">{</span>
<span class="c1">// Taking screenshot after test</span>
<span class="k">let</span> <span class="nv">screenshot</span> <span class="o">=</span> <span class="kt">XCUIScreen</span><span class="o">.</span><span class="n">main</span><span class="o">.</span><span class="nf">screenshot</span><span class="p">()</span>
<span class="k">let</span> <span class="nv">fullScreenshotAttachment</span> <span class="o">=</span> <span class="kt">XCTAttachment</span><span class="p">(</span><span class="nv">screenshot</span><span class="p">:</span> <span class="n">screenshot</span><span class="p">)</span>
<span class="n">fullScreenshotAttachment</span><span class="o">.</span><span class="n">lifetime</span> <span class="o">=</span> <span class="o">.</span><span class="n">keepAlways</span>
<span class="nf">add</span><span class="p">(</span><span class="n">fullScreenshotAttachment</span><span class="p">)</span>
<span class="c1">// Closing the app</span>
<span class="n">app</span><span class="o">.</span><span class="nf">terminate</span><span class="p">()</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>This test is looking for the text field at first, then inputs some text into it, presses the button and waits for alert to appear. When alert appears, it will verify its text.</p>
<p class="notice--info">There was a basic wait used in the code snippet above. If you want to find more about advanced techniques of doing that, check out my recent post about <a href="http://alexilyenko.github.io/xcuitest-waiting/">Waits in XCUITest</a>.</p>
<p>That’s it for today’s XCTest UI Testing Basics. If you have something to add please share it in the comments.</p>
<p><a href="https://alexilyenko.github.io/"><img src="https://alexilyenko.github.io/assets/images/share_message.png" alt="Feel free to share!" /></a></p>Alex IlyenkoEverything you need to know to start using XCUITest in iOS functional testingParallel Android Tests with Spoon2017-09-15T00:00:00-04:002017-09-15T00:00:00-04:00https://alexilyenko.github.io/android-parallel<aside class="sidebar__right">
<nav class="toc">
<header><h4 class="nav__title"><i class="fas fa-file-text"></i> Parallel Android Tests with Spoon</h4></header>
<ul class="toc__menu" id="markdown-toc">
<li><a href="#why-spoon" id="markdown-toc-why-spoon">Why Spoon?</a></li>
<li><a href="#setup" id="markdown-toc-setup">Setup</a></li>
<li><a href="#parallel-run" id="markdown-toc-parallel-run">Parallel run</a></li>
<li><a href="#test-reports" id="markdown-toc-test-reports">Test reports</a></li>
<li><a href="#screenshots" id="markdown-toc-screenshots">Screenshots</a></li>
</ul>
</nav>
</aside>
<h2 id="why-spoon">Why Spoon?</h2>
<p>It’s a well known fact that Mobile Apps are the next big thing and no one argues their testing is really important nowadays. There are couple of open source tools allowing to run your functional mobile tests across multiple platforms in parallel to ensure the app runs as expected on different sets of OS versions, screen sizes and hardware configurations. But what if your application’s supposed to run only on one of those set?</p>
<figure class="">
<img src="/assets/images/sharding.gif" alt="Parallel test run on different Android devices" /><figcaption>
Parallel test run on different Android devices
</figcaption></figure>
<p>Let’s imaging hypothetical situation. There is a team out there who’s developing FireTV application. For the moment there is only one versions of FireTV devices on the market. Test developers from the team implemented 100 functional UI tests. They’ve got 10 TV devices for testing purposes. Now, if they ran their tests with native instruments (i.e. UiAutomator and JUnit), they would get hundred of tests running on each of the 10 devices in parallel, which is basically waste of resources. Hence to get the most benefits from their amount of devices they would need to <strong>shard</strong>* those 100 tests to all of their devices. That way each of the them will get 10 tests and total time of execution might be decreased by a factor of 10 (in ideal case).</p>
<p class="notice--info"><strong>Sharding</strong>* is splitting the test suite into multiple threads, so they can be easily run as a group under the same Instrumentation instance and on the same device.</p>
<p>So how do we shard our tests with native tools like JUnit and <a href="https://alexilyenko.github.io/uiautomator-basics/">UiAutomator</a>? I would recommend <a href="http://square.github.io/spoon/">Spoon</a> for this. It’s easy-to-use lightweight tool, which main goal is to distribute execution of the tests, making their output more useful. Spoon will help you to shard and run your tests on multiple devices simultaneously with no-brainer. Let’s find out how to setup this useful tool.</p>
<h2 id="setup">Setup</h2>
<p>Since we decided to leverage native Android tools for our testing and Gradle is the standard de-facto for all Android projects, we will use <a href="https://github.com/stanfy/spoon-gradle-plugin">Spoon Gradle Plugin</a>.</p>
<p>The first step would be the adding Spoon’s class path to top-level <code class="language-plaintext highlighter-rouge">build.gradle</code> file:</p>
<div class="language-groovy highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">buildscript</span> <span class="o">{</span>
<span class="c1">// your configurations go here </span>
<span class="n">dependencies</span> <span class="o">{</span>
<span class="c1">// other dependencies go here</span>
<span class="n">classpath</span> <span class="s1">'com.stanfy.spoon:spoon-gradle-plugin:1.2.2'</span>
<span class="o">}</span>
<span class="o">}</span>
</code></pre></div></div>
<p>The next thing to do is to configure app’s <code class="language-plaintext highlighter-rouge">build.gradle</code>. At first we will apply Spoon plugin and then configure Spoon task.</p>
<div class="language-groovy highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">apply</span> <span class="nl">plugin:</span> <span class="s1">'spoon'</span>
<span class="c1">// rest of your configuration</span>
<span class="n">spoon</span> <span class="o">{</span>
<span class="n">shard</span> <span class="o">=</span> <span class="kc">true</span>
<span class="o">}</span>
</code></pre></div></div>
<p><code class="language-plaintext highlighter-rouge">shard=true</code> means that Spoon will use its built-in auto sharding function and we won’t need to care about splitting our tests manually. More available settings can be found in the <a href="https://github.com/stanfy/spoon-gradle-plugin/blob/master/README.md">official documentation</a>.</p>
<p>Now let’s check if we set up everything correctly. Run <code class="language-plaintext highlighter-rouge">./gradlew tasks</code> under your project folder. We should be able to see Spoon’s tasks under Verification group, like it’s shown in the figure below:</p>
<figure>
<a href="https://alexilyenko.github.io/assets/images/spoon_tasks.png"><img src="https://alexilyenko.github.io/assets/images/spoon_tasks.png" /></a>
<figcaption>Verify if Spoon tasks are available under Verification group</figcaption>
</figure>
<p>Assuming we were able to see those tasks, we’re ready to run our tests on multiple devices in parallel!</p>
<h2 id="parallel-run">Parallel run</h2>
<p>Spoon is designed as a tool for running all of your functional tests across all of the connected devices (i.e. visible by <code class="language-plaintext highlighter-rouge">adb devices</code> command). Thus if you plug in multiple different phones, tablets, <a href="https://alexilyenko.github.io/androidtv-automated-tests/">TVs</a> and start various emulators, you’ll get great diversity of the targets.</p>
<p class="notice--info">Be sure to unplug all not needed for tests devices and shutdown all non-target emulators before executing Spoon’s tasks.</p>
<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code>./gradlew spoon
</code></pre></div></div>
<p>At this point our tests will be sharded across all connected Android devices and emulators.</p>
<figure class="">
<img src="/assets/images/parallel.gif" alt="Spoon test sharding on Android emulators" /><figcaption>
Spoon test sharding on Android emulators
</figcaption></figure>
<h2 id="test-reports">Test reports</h2>
<p>Besides being convenient and easy-to-use tool for parallel test run, Spoon’s also got some additional functionality, which might be useful for Test Developers.</p>
<p>Once all tests have been completed, an HTML report will be generated with detailed info about tests and devices they were run on. Thanks to the high level data in reports we can determine right-away if the failure is specific to some device or to all of them. Also it provides device logs and optional screenshots.</p>
<figure class="">
<img src="/assets/images/device_view.png" alt="Parallel Android tests with Spoon" /><figcaption>
Device View
</figcaption></figure>
<figure class="">
<img src="/assets/images/test_view.png" alt="Spoon test sharding on Android emulators" /><figcaption>
Test View
</figcaption></figure>
<h2 id="screenshots">Screenshots</h2>
<p>As I said before Spoon allows us to snap screenshots at any points while test is executed. Later they will be automatically included into test report HTML. This can visualize test execution and point to the potential bugs in your app.</p>
<p>To be able to take screenshots we’ll need to include Spoon dependency to app’s <code class="language-plaintext highlighter-rouge">build.gradle</code>:</p>
<div class="language-groovy highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">dependencies</span> <span class="o">{</span>
<span class="n">androidTestCompile</span> <span class="s1">'com.squareup.spoon:spoon-client:1.7.1'</span>
<span class="o">}</span>
</code></pre></div></div>
<p>Furthermore since Spoon stores all taken screenshot in the device’s memory card, we’ll have to enable write storage privileges for our test app. This can be done in <code class="language-plaintext highlighter-rouge">AndroidManifest.xml</code>, which can be found under <code class="language-plaintext highlighter-rouge">main</code> folder in your project directory.</p>
<div class="language-xml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nt"><manifest</span> <span class="na">xmlns:android=</span><span class="s">"http://schemas.android.com/apk/res/android"</span>
<span class="na">xmlns:tools=</span><span class="s">"http://schemas.android.com/tools"</span>
<span class="na">package=</span><span class="s">"io.github.alexilyenko.sample"</span><span class="nt">></span>
<span class="nt"><uses-permission</span> <span class="na">android:name=</span><span class="s">"android.permission.WRITE_EXTERNAL_STORAGE"</span> <span class="nt">/></span>
<span class="c"><!-- Rest of the Manifest goes below --></span>
<span class="nt"></manifest></span>
</code></pre></div></div>
<p>Now all we have to do is to invoke static method <code class="language-plaintext highlighter-rouge">Spoon#screenshot</code> in our tests like this:</p>
<div class="language-kotlin highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nc">Spoon</span><span class="p">.</span><span class="nf">screenshot</span><span class="p">(</span><span class="n">activity</span><span class="p">,</span> <span class="n">screenshotName</span><span class="p">,</span> <span class="n">testClassName</span><span class="p">,</span> <span class="n">testMethodName</span><span class="p">)</span>
</code></pre></div></div>
<p>Full example of invocation along with the retrieval of current activity can be found in <a href="https://github.com/alexilyenko/SimpleAndroidCalculator/blob/master/app/src/androidTest/kotlin/io/github/alexilyenko/sample/pageobject/tests/BaseTest.kt">GitHub repository</a>. All examples are written in Kotlin, hence if you haven’t heard about this programming language I recommend to read my post about <a href="https://alexilyenko.github.io/kotlin-android-tests/">Android Tests in Kotlin</a>.</p>
<p><a href="https://alexilyenko.github.io/"><img src="https://alexilyenko.github.io/assets/images/share_message.png" alt="Feel free to share!" /></a></p>Alex IlyenkoLearn how to run your UI tests on multiple devices simultaneouslyHow to test Android TV apps2017-09-10T00:00:00-04:002017-09-10T00:00:00-04:00https://alexilyenko.github.io/androidtv-automated-tests<h2 id="why-do-we-need-to-test-android-tv-app">Why do we need to test Android TV app</h2>
<p>After Google introduced <a href="https://www.android.com/tv/">Android TV</a> in 2014 and Apple released its <a href="https://www.apple.com/tvos/">tvOS</a> in 2015 it became clear that there is an unseen market with great demand coming from our living room. Even though it’s still a young one, almost every company is trying to release its own TV app as soon as possible. It’s quiet understandable because as they say “the first man gets the oyster, the second man gets the shell”.</p>
<p>The TV app development is similar to the mobile development in many ways. Thus if you want to deliver high quality product to your target audience, you need to do comprehensive testing and to use the same QA processes as you would leverage for mobile app development. I won’t go into details of designing testing strategy for TV apps in this post - it’s completely up to you and your teammates. Instead, I’ll try to explain the key concepts of building automated infrastructure for Android TV app functional testing.</p>
<p class="notice--info">You may also want to check out one of my other posts about <a href="https://alexilyenko.github.io/apple-tv-automated-tests/">Automated Testing for Apple TV apps</a></p>
<h2 id="what-tool-to-use-for-android-tv-testing">What tool to use for Android TV testing</h2>
<p>As I said before Android TV development is more or less similar to usual Android development. The same applies to the testing. You don’t need to invent a tool for that, there have already been wide range of them on the market.</p>
<p>The list of the tools for Android TV testing:</p>
<ul>
<li>Appium</li>
<li>Espresso</li>
<li>Robotium</li>
<li>Calabash</li>
<li>UiAutomator</li>
</ul>
<figure class="">
<img src="/assets/images/tools.png" alt="Variety of the tools for Android TV testing" /><figcaption>
Variety of the tools for Android TV testing
</figcaption></figure>
<p>Each tool has its own advantages and limitations but in this post I will be talking about the last one - <strong>UiAutomator</strong>. There are couple of reasons I prefer it over the others. It’s the native instrument, provided by Google, most of the developers are familiar with it and almost all automation tools are built on top of it. If you haven’t heard about it yet, you can check out my post about <a href="https://alexilyenko.github.io/uiautomator-basics/">UiAutomator Essentials</a>.</p>
<h2 id="differences-from-mobile-apps">Differences from Mobile apps</h2>
<p>On the lowest level mobile functional tests consist of finding elements, clicking/swiping/drag-and-dropping on them, changing or retrieving their attributes and verifying outputs. The same idea applicable to Android TV tests. There are some platform specifics though:</p>
<ul>
<li>Devices do not have touchscreen</li>
<li>All actions are done using Remote control</li>
<li>Screen is in landscape orientation only</li>
<li>Most apps are aimed to video playback</li>
</ul>
<p>The first two bullets are the ones which might cause difficulties in designing the framework. But I’ll try to explain dealing with them in further reading.</p>
<h2 id="navigating-through-android-tv-app">Navigating through Android TV app</h2>
<p>Since Android TV devices do not have touchscreens the navigation should be done via special device - Remote controller.</p>
<figure class="">
<img src="/assets/images/remote.jpg" alt="Default Android TV Remote Controller" /><figcaption>
Android TV Remote Controller Example
</figcaption></figure>
<p>This is the first thing we have to consider while starting building our automation framework. UiAutomator allows us to use Remote in the way we want, providing a command for every particular interaction:</p>
<div class="language-kotlin highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">fun</span> <span class="nf">back</span><span class="p">()</span> <span class="p">=</span> <span class="n">device</span><span class="p">.</span><span class="nf">pressBack</span><span class="p">()</span>
<span class="k">fun</span> <span class="nf">menu</span><span class="p">()</span> <span class="p">=</span> <span class="n">device</span><span class="p">.</span><span class="nf">pressMenu</span><span class="p">()</span>
<span class="k">fun</span> <span class="nf">home</span><span class="p">()</span> <span class="p">=</span> <span class="n">device</span><span class="p">.</span><span class="nf">pressHome</span><span class="p">()</span>
<span class="k">fun</span> <span class="nf">playPause</span><span class="p">()</span> <span class="p">=</span> <span class="n">device</span><span class="p">.</span><span class="nf">pressKeyCode</span><span class="p">(</span><span class="nc">KeyEvent</span><span class="p">.</span><span class="nc">KEYCODE_MEDIA_PLAY_PAUSE</span><span class="p">)</span>
<span class="c1">// D-Pad Controls:</span>
<span class="k">fun</span> <span class="nf">left</span><span class="p">()</span> <span class="p">=</span> <span class="n">device</span><span class="p">.</span><span class="nf">pressDPadLeft</span><span class="p">()</span>
<span class="k">fun</span> <span class="nf">right</span><span class="p">()</span> <span class="p">=</span> <span class="n">device</span><span class="p">.</span><span class="nf">pressDPadRight</span><span class="p">()</span>
<span class="k">fun</span> <span class="nf">down</span><span class="p">()</span> <span class="p">=</span> <span class="n">device</span><span class="p">.</span><span class="nf">pressDPadDown</span><span class="p">()</span>
<span class="k">fun</span> <span class="nf">up</span><span class="p">()</span> <span class="p">=</span> <span class="n">device</span><span class="p">.</span><span class="nf">pressDPadUp</span><span class="p">()</span>
<span class="k">fun</span> <span class="nf">select</span><span class="p">()</span> <span class="p">=</span> <span class="n">device</span><span class="p">.</span><span class="nf">pressDPadCenter</span><span class="p">()</span>
</code></pre></div></div>
<p class="notice--info"><strong>Note</strong>: code snippets given in this post are written in Kotlin. If you want to start using this powerful programming language in your automation too, make sure you read this post about <a href="https://alexilyenko.github.io/kotlin-android-tests/">Kotlin in Android tests</a></p>
<p>Now when we know how to navigate in Android TV app, the next question would be “how to select elements we need?”. The answer is - the same way as we would do in manual testing. We’d need to find element, move focus to it and select it.</p>
<figure class="">
<img src="/assets/images/navigation.gif" alt="Navigating through Android TV app" /><figcaption>
Navigating through Android TV app
</figcaption></figure>
<p>That said for successful navigation we’ll have to implement several additional methods for determination of focus state:</p>
<ul>
<li><code class="language-plaintext highlighter-rouge">focused</code> returns currently focused element
<div class="language-kotlin highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">val</span> <span class="py">focused</span><span class="p">:</span> <span class="nc">UiObject2</span><span class="p">?</span> <span class="k">get</span><span class="p">()</span> <span class="p">=</span>
<span class="n">device</span><span class="p">.</span><span class="nf">findObject</span><span class="p">(</span><span class="nc">By</span><span class="p">.</span><span class="nf">focused</span><span class="p">(</span><span class="k">true</span><span class="p">))</span>
</code></pre></div> </div>
</li>
<li><code class="language-plaintext highlighter-rouge">isFocused</code> checks if particular element is focused
<div class="language-kotlin highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">fun</span> <span class="nf">isFocused</span><span class="p">(</span><span class="n">selector</span><span class="p">:</span> <span class="nc">BySelector</span><span class="p">):</span> <span class="nc">Boolean</span> <span class="p">=</span>
<span class="n">device</span><span class="p">.</span><span class="nf">findObject</span><span class="p">(</span><span class="n">selector</span><span class="p">)</span><span class="o">?.</span><span class="n">isFocused</span> <span class="o">?:</span> <span class="k">false</span>
</code></pre></div> </div>
</li>
<li>Some elements are not focusable, but they might contain focused element inside them. <code class="language-plaintext highlighter-rouge">hasFocus</code> will help us in those cases:
<div class="language-kotlin highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">fun</span> <span class="nf">hasFocus</span><span class="p">(</span><span class="n">selector</span><span class="p">:</span> <span class="nc">BySelector</span><span class="p">):</span> <span class="nc">Boolean</span> <span class="p">=</span>
<span class="n">device</span><span class="p">.</span><span class="nf">findObject</span><span class="p">(</span><span class="n">selector</span><span class="p">.</span><span class="nf">hasDescendant</span><span class="p">(</span><span class="nc">By</span><span class="p">.</span><span class="nf">focused</span><span class="p">(</span><span class="k">true</span><span class="p">)))</span>
<span class="p">!=</span> <span class="k">null</span>
</code></pre></div> </div>
</li>
</ul>
<p>Hence to select element we’ll need to move focus onto it first. Now, this part might be tricky in terms of automation. In other words we have to build focus movement path to the element. The more optimal path is the faster and more efficient is navigation through the app. Let’s discuss different approaches for finding path to the element.</p>
<h2 id="strategies-for-finding-elements-in-android-tv-app">Strategies for finding elements in Android TV app</h2>
<h3 id="finding-element-in-the-column">Finding element in the column</h3>
<p>We’ll start from the easiest approach - finding element in column. Columns in Android TV apps are usually represented by menus of different kinds.</p>
<figure class="">
<img src="/assets/images/column.gif" alt="Finding element in column in Android TV" /><figcaption>
Finding element in column
</figcaption></figure>
<p>While searching for element in the column we should keep in mind couple of things:</p>
<ol>
<li>We can move either up or down only</li>
<li>Element may be already focused</li>
<li>Element might be located either above or below currently focused element</li>
<li>We need to determine if the end of the column is reached</li>
<li>Element might not be in the column at all</li>
</ol>
<p>Considering all of the above we could build simple algorithm for finding element in the column of Android TV app:</p>
<div class="language-kotlin highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">fun</span> <span class="nf">findInColumn</span><span class="p">(</span><span class="k">by</span><span class="p">:</span> <span class="nc">BySelector</span><span class="p">)</span> <span class="p">{</span>
<span class="kd">var</span> <span class="py">isEndReached</span> <span class="p">=</span> <span class="k">false</span>
<span class="k">while</span> <span class="p">(!</span><span class="nf">isFocused</span><span class="p">(</span><span class="k">by</span><span class="p">))</span> <span class="p">{</span>
<span class="kd">val</span> <span class="py">previous</span><span class="p">:</span> <span class="nc">UiObject2</span><span class="p">?</span> <span class="p">=</span> <span class="n">focused</span>
<span class="nf">moveFocus</span><span class="p">(</span><span class="n">isEndReached</span><span class="p">)</span>
<span class="kd">val</span> <span class="py">current</span><span class="p">:</span> <span class="nc">UiObject2</span><span class="p">?</span> <span class="p">=</span> <span class="n">focused</span>
<span class="k">if</span> <span class="p">(</span><span class="n">previous</span> <span class="p">==</span> <span class="n">current</span><span class="p">)</span> <span class="p">{</span>
<span class="k">if</span> <span class="p">(</span><span class="n">isEndReached</span><span class="p">)</span> <span class="p">{</span>
<span class="k">throw</span> <span class="nc">RuntimeException</span><span class="p">(</span><span class="s">"Element $by was not found"</span><span class="p">)</span>
<span class="p">}</span>
<span class="n">isEndReached</span> <span class="p">=</span> <span class="k">true</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="nf">println</span><span class="p">(</span><span class="s">"Element $by was found"</span><span class="p">)</span>
<span class="nf">select</span><span class="p">()</span>
<span class="p">}</span>
<span class="k">fun</span> <span class="nf">moveFocus</span><span class="p">(</span><span class="n">isEndReached</span><span class="p">:</span> <span class="nc">Boolean</span><span class="p">)</span> <span class="p">=</span>
<span class="k">if</span> <span class="p">(</span><span class="n">isEndReached</span><span class="p">)</span> <span class="nf">up</span><span class="p">()</span> <span class="k">else</span> <span class="nf">down</span><span class="p">()</span>
</code></pre></div></div>
<p>We are moving through the elements in the column checking if given element is focused. While doing that we check if the end of the column reached by comparing previous focused element with current one. If end is reached for the second time, which means all elements have been inspected, exception will be thrown.</p>
<h3 id="finding-element-in-the-row">Finding element in the row</h3>
<p>Finding element in row is essential for the testing of Android TV. It’s so important because almost every app keeps its content in so called “shelves”. And user can navigate easily from one element to another the same way she would do in the real-world video store.</p>
<figure class="">
<img src="/assets/images/row.gif" alt="Finding element in row in Android TV" /><figcaption>
Finding element in row
</figcaption></figure>
<p>For finding element in the row we could use the same principles as for column look up. Actually our algorithm will look almost the same. The only thing we’d need to change is <code class="language-plaintext highlighter-rouge">moveFocus</code> method:</p>
<div class="language-kotlin highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">fun</span> <span class="nf">moveFocus</span><span class="p">(</span><span class="n">isEndReached</span><span class="p">:</span> <span class="nc">Boolean</span><span class="p">)</span> <span class="p">=</span>
<span class="k">if</span> <span class="p">(</span><span class="n">isEndReached</span><span class="p">)</span> <span class="nf">left</span><span class="p">()</span> <span class="k">else</span> <span class="nf">right</span><span class="p">()</span>
</code></pre></div></div>
<p>This way we will inspect elements to both - left and right sides from the currently focused element.</p>
<h3 id="finding-element-in-the-grid">Finding element in the grid</h3>
<p>Vertical grid is the most difficult place to find elements because we have to move in three directions - left, right and down.</p>
<iframe width="560" height="315" src="https://www.youtube.com/embed/s9FT2fW6M94?rel=0" frameborder="0" allowfullscreen=""></iframe>
<p>Several things to consider before actual search:</p>
<ol>
<li>The best place to start is the upper left/right element. Starting from it we won’t need to return to check if element was missed in the beginning</li>
<li>Since in our case grid’s row is not completely visible, all elements in row should be inspected before switching to next one</li>
<li>In the end of each row direction should be changed too</li>
<li>Grid can be asymmetrical (last row is incomplete), which means we have to inspect both of its bottom elements</li>
</ol>
<p>Schematically our algorithm would work like this:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>1 -> 2 -> 3 -> 4 -> 5
|
v
6 <- 7 <- 8 <- 9 <- 10
|
v
11 -> 12 -> 13 -> 14 -> 15
<- <- <- <-
|
v
16 -> 17 -> 18
</code></pre></div></div>
<p>Schema example represents asymmetrical grid on purpose. When 15th element is reached(<code class="language-plaintext highlighter-rouge">isEndReached = true</code>), we will return to the 11th one and go down to search element in the last incomplete row.</p>
<p>Actual implementation would look like this:</p>
<div class="language-kotlin highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">fun</span> <span class="nf">findInGrid</span><span class="p">(</span><span class="k">by</span><span class="p">:</span> <span class="nc">BySelector</span><span class="p">)</span> <span class="p">{</span>
<span class="kd">var</span> <span class="py">isMovingRight</span> <span class="p">=</span> <span class="k">true</span>
<span class="kd">var</span> <span class="py">previous</span><span class="p">:</span> <span class="nc">UiObject2</span><span class="p">?</span> <span class="p">=</span> <span class="k">null</span>
<span class="kd">var</span> <span class="py">current</span><span class="p">:</span> <span class="nc">UiObject2</span><span class="p">?</span>
<span class="kd">var</span> <span class="py">isEndReached</span> <span class="p">=</span> <span class="k">false</span>
<span class="k">while</span> <span class="p">(!</span><span class="nf">isFocused</span><span class="p">(</span><span class="k">by</span><span class="p">))</span> <span class="p">{</span>
<span class="nf">moveFocus</span><span class="p">(</span><span class="n">isMovingRight</span><span class="p">)</span>
<span class="n">current</span> <span class="p">=</span> <span class="n">focused</span>
<span class="k">if</span> <span class="p">(</span><span class="n">current</span> <span class="p">==</span> <span class="n">previous</span><span class="p">)</span> <span class="p">{</span>
<span class="nf">down</span><span class="p">()</span>
<span class="n">isMovingRight</span> <span class="p">=</span> <span class="p">!</span><span class="n">isMovingRight</span>
<span class="n">current</span> <span class="p">=</span> <span class="n">focused</span>
<span class="k">if</span> <span class="p">(</span><span class="n">current</span> <span class="p">==</span> <span class="n">previous</span><span class="p">)</span> <span class="p">{</span>
<span class="k">if</span> <span class="p">(</span><span class="n">isEndReached</span><span class="p">)</span> <span class="p">{</span>
<span class="k">throw</span> <span class="nc">RuntimeException</span><span class="p">(</span><span class="s">"Element $by was not found!"</span><span class="p">)</span>
<span class="p">}</span> <span class="k">else</span> <span class="p">{</span>
<span class="n">isEndReached</span> <span class="p">=</span> <span class="k">true</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="n">previous</span> <span class="p">=</span> <span class="n">current</span>
<span class="p">}</span>
<span class="nf">println</span><span class="p">(</span><span class="s">"Element $by was found!"</span><span class="p">)</span>
<span class="nf">select</span><span class="p">()</span>
<span class="p">}</span>
<span class="k">fun</span> <span class="nf">moveFocus</span><span class="p">(</span><span class="n">isMovingRight</span><span class="p">:</span> <span class="nc">Boolean</span><span class="p">)</span> <span class="p">=</span>
<span class="k">if</span> <span class="p">(</span><span class="n">isMovingRight</span><span class="p">)</span> <span class="nf">right</span><span class="p">()</span> <span class="k">else</span> <span class="nf">left</span><span class="p">()</span>
</code></pre></div></div>
<p>The idea is the same we’ve seen in <code class="language-plaintext highlighter-rouge">findInColumn</code>. The key difference is switching from row to row when end of the first one is reached. Also test developer should foresee the case wherever grid is asymmetrical, inspecting both sides of the last row before throwing the exception.</p>
<h3 id="further-optimizations">Further optimizations</h3>
<p>All algorithms in the post could be optimized. For instance, I prefer to increase number of steps of focus movement while element is not visible to speed up searching.</p>
<p>Also part of the code from each approach can be reused. As a matter of fact, I’m using the same method for all three strategies and differentiating them by created <code class="language-plaintext highlighter-rouge">Direction</code> enums.</p>
<p>As for grid search, whenever you are able to see the whole row, you can skip rows until you spot the particular element.</p>
<p>If you’re wondering how to build-in these approaches into your framework, I suggest to move “find methods” to <code class="language-plaintext highlighter-rouge">BaseScreen</code> class or even create dedicated <code class="language-plaintext highlighter-rouge">Actions</code> class for encapsulating all device actions. More on doing that can be found in my post - <a href="https://alexilyenko.github.io/uiautomator-page-object/">Page Object in designing test framework with UiAutomator</a>.</p>
<p>Also the good practice is to run your tests on multiple devices simultaneously. It will help you to get faster feedback about any potential issues and thus decrease amount of resources needed for fixing it. If that’s what you need, you could read my article about <a href="https://alexilyenko.github.io/android-parallel/">Parallel Functional Android Tests</a>.</p>
<p>In case you have other thoughts on optimization I’d love to hear them!</p>
<p><a href="https://alexilyenko.github.io/"><img src="https://alexilyenko.github.io/assets/images/share_message.png" alt="Feel free to share!" /></a></p>Alex IlyenkoLearn how to implement automated tests for Android TV