Getting started with Maestro Mobile UI Testing Framework

Selecting the right thing can become a difficult decision, especially with a large amount of offers. Quality assurance engineers know the struggle well, as this is also applicable to test automation tool selection.

There might be different reasons why you would like to start looking for a test automation tool. Maybe you want to change the automation tool to reduce costs or made a wrong choice and now experience scalability issues, or decided to add automation to existing manual tests. Whatever the reason, choosing the right one won’t be easy. Nevertheless, the more tools you know about, the higher chances that you will find a perfect match. So let’s stay up-to-date and take a look at a recently appeared open-source mobile UI testing framework called Maestro. 

Maestro appeared on the market with its first release in July 2022 and positioned itself as the simplest and most effective mobile UI testing framework. Maestro allows automation of end-to-end scenarios on major development platforms such as iOS, Android, React Native, Flutter, and also Web Views on both iOS and Android. In comparison to most automation tools, Maestro doesn’t require knowledge of a  complicated programming language. It uses a human-readable YAML language that is stored in .yaml files. Yet, Maestro allows to inject JavaScript code and to make HTTP requests.

In this article, we are going to install Maestro, run a test flow, explore available commands, try JavaScript injection, and learn about Maestro Studio and Maestro Cloud.

Start with Maestro

Prepare an environment

One of the advantages of Maestro is that it can be installed on macOS, Windows, or Linux OS with a single command. However, Windows OS users will have to fulfill additional pre-conditions including setting up Windows Subsystem for Linux (WSL). Besides that, to work effectively you need to have the possibility to execute tests locally. That requires additional environments and tools installation.

To run test flows locally on the iOS simulator: 

• Install Xcode development environment
• Make sure Command Line Tools are installed and displayed in Xcode > Settings > Locations > Command Line Tools. If not, run a command in the Terminal:

xcode-select –install

• Install Facebook IDB tool by running two commands:

brew tap facebook/fb
brew install facebook/fb/idb-companion

To run test flows locally on Android emulator install Android Studio development environment.

• If you are planning to test APKs on a physical device you might also want to get adb command line to install application from the Terminal

The list doesn’t mention iOS physical devices as they are not supported. That limits cross platform testing possibilities on Windows and Linux operating systems and might cause increased CPU consumption on macOS produced by simulator usage. 

Windows OS users should brace themselves and follow the guide to fulfill all the pre-conditions.

Install Maestro

To install Maestro framework run a command in the command line:

curl -Ls "https://get.maestro.mobile.dev" | bash

Verify version with a command:

maestro --version

Run the app

Once installation is completed, we are one step closer to the testing activity. Now let’s build and install the application that we are going to test. Currently Maestro supports:

1. Android emulator (.apk)
2. Physical Android device (.apk)
   (Note: To execute Maestro flows on a device, connect the device via USB port and make sure USB debugging is enabled.)
3. iOS simulator (.app)

For the testing purpose let’s use a publicly available Wikipedia application that can be downloaded from Maestro documentation here.

To install .apk on an emulator or a physical device:

1. Download Wikipedia application from the provided link
2. Boot Android emulator in AndroidStudio or connect the device via USB
3. Launch Terminal in the folder with .apk file
4. Run a command:

adb install sample.apk

The steps to install the application on an iOS simulator are:

1. Download Wikipedia application from the provided link
2. Boot iOS simulator in Xcode
3. Launch Terminal in the folder with .app file
4. Run a command:

xcrun simctl install Booted Wikipedia.app

This command will install the application. We are ready to proceed now.

Running a first flow

Since everything is up and running we can start testing, so let’s write our first flow. We are going to write commands to launch the app, skip onboarding, make a search, find and select from suggested, and verify text visibility on the opened page. The flow itself can be written directly in a YAML file or using Maestro Studio, which we will talk about later, but let’s do it an old fashioned way first.

Step 1: Create a new file with  .yaml extension

Step 2: Open the file and indicate which application to test:

appId: org.wikimedia.wikipedia
---

Required ID is called “package ID” for Android applications and “bundle ID” for iOS. 

Where to find the ID:

• When Android device is connected via USB and debugging is enabled, ID can be found in the list of all installed packages: 

adb shell pm list packages | grep "App name"

• In the app’s page URL in Google Play Store

In Xcode’s: Project navigator > Project folder > General > Identify

Step 3: Launch the app with a command:

- launchApp

Step 4: Tap on the element with the text “Skip” to skip onboarding:

- tapOn: "Skip"

Step 5: Tap on the text “Search Wikipedia” to set a focus:

- tapOn:
  text: "Search Wikipedia"

Step 6: Input “User” text into the focused search field:

- inputText: "User"

Step 7: Scroll down a list of suggested results until "User interface" text is visible:

- scrollUntilVisible:
  element: "User interface"
  direction: DOWN

Step 8: Tap on “User interface” text in the suggested results:

- tapOn: "User interface"

Step 9: Assert that a text line starting with “Means by which a user interacts .*" is visible on the page:

- assertVisible: "Means by which a user interacts .*"

Please note that the used sign combination “.*” is a wildcard

Step 10: Save the changes. The full flow looks like that:

appId: org.wikimedia.wikipedia
---
- launchApp
- tapOn: "Skip"
- tapOn:
   text: "Search Wikipedia"
- inputText: "User"
- scrollUntilVisible:
   element: "User interface"
   direction: DOWN
- tapOn: "User interface"
- assertVisible: "Means by which a user interacts .*"

Step 11: Open Terminal in the file location and run the flow:

maestro test fileName.yaml

Step 12: Observe the testing process on the device and commands execution in the Terminal

We just effortlessly created and executed an End-to-End test. The commands used in the flow are straightforward and easy to remember. Such tests can be created without additional actions, only by using the text from the elements. But how to interact with elements that don't have a text? One of the options is to use a command:

maestro hierarchy

It will print out a screen hierarchy of the current state of the application. It will show different useful information, including attribute ID:

"attributes" : {
 "text" : "",
 "accessibilityText" : "Back",
 "hintText" : "",
 "ignoreBoundsFiltering" : "false",
 "resource-id" : "com.android.systemui:id/back",
 "clickable" : "true",
 "bounds" : "[219,2218][399,2340]",
 "enabled" : "true",
 "focused" : "false",
 "checked" : "false",
 "selected" : "false"
}

Another option is to use Maestro Studio. Let’s find out what it is.

Maestro Studio

Maestro Studio is a tool that helps create testing flows. There are many reasons why you would want to choose Maestro Studio overwriting tests directly in a YAML file. Besides the visually pleasing GUI, in the Maestro Studio you will be able to:

• Interact with the app
• Find element’s ID
• Search elements
• Write commands
• Automatically generate commands
• Execute commands separately without running the whole flow
• Copy executed commands
• Export flow in a YAML file

It is a built-in Maestro CLI tool and to launch it you just need execute a single command:

maestro studio

It will launch in a browser window and present a view of the running device and a command line. For better clarity, let’s take a look on example of how to create a testing flow in Maestro Studio:

Extremely easy and quick I must say. With such an intuitive GUI and automatically generated examples, the flow can be created by a person without any programming skills or technical knowledge. That could be a way for manual QAs to start their journey in the world of automation.

Maestro Cloud

Locally running emulators and simulators require a lot of processing power, while Maestro provides an alternative way to execute flows on Maestro Cloud.

If you have Maestro installed, you don’t need any additional installations to run tests remotely on the Cloud. You just need to:

1. Create an account or log in to existing account

maestro login

2. Run the flow on the Cloud

maestro cloud sample.apk test.yaml

Execution results can be observed on Cloud Console. Results are not limited to the “pass” or “fail” status, but rather provide a full picture with the possibility to:

• Check test result
• View and download a workspace
• Download screenshots
• View test commands, hierarchy and logs
• View test screen recording
• and many more

Furthermore, Maestro Cloud can be integrated with any CI platform. In a project with multiple contributors, that practice is beneficial and provides efficiency to the automation test development process.

Commands review

Maestro offers a list of commands that can be used for the automation. The amount of commands, properties and methods increases with new releases. Enhanced variety helps to improve existing flows, add new verifications and adjust tests to environmental peculiarities. 

Here are available commands and their short descriptions:

- launchApp: # Launches the app under test
- stopApp: # Stops current application
- tapOn: # Taps on a selected element
- doubleTapOn: # Double taps on a selected element
- inputText: # Inputs text
- eraseText: # Removes characters from the currently selected text field
- assertVisible: # Asserts whether an element is visible
- assertNotVisible: # Asserts whether an element is not visible
- copyTextFrom: # Copies text from an element and saves it in-memory
- pasteText: # Pastes any text copied with copyTextFrom into the currently focused field
- startRecording: # Starts a screen recording
- stopRecording: # Stops a running screen recording
- scroll: # Does a vertical scroll
- scrollUntilVisible: # Scrolls towards a direction until an element becomes visible in the view hierarchy
- waitForAnimationToEnd: # Waits until an ongoing animation/video is fully finished, and the screen becomes static
- extendedWaitUntil: # Waits until an element becomes visible within a specified amount of time
- back: # Navigates the user to the previous screen
- pressKey: # Presses a set of special keys
- runFlow: # Runs commands from another file
- runScript: # Runs a provided JavaScript file
- setLocation: # Applies a mock geolocation to the device
- travel: # Mocks the motion of the user, by specifying a set of points and a speed
- clearState: # Clears the application state
- clearKeychain: # Clears the entire iOS keychain
- hideKeyboard: # Hides the software keyboard
- swipe: # Makes the swipe gesture
- addMedia: # Adds media to the device’s gallery
- takeScreenshot: # Saves a screenshot in a PNG file
- assertTrue: # Asserts whether the given value is either true or non-empty
- repeat: # Repeats set of commands N times
- openLink: # Opens a link on a device
- evalScript: # Allows specifying JavaScript directly in the Maestro flow

Most of the commands are self-explanatory and don't really require an explanatory description. Such command names are easy to remember and use. Additionally, from some of the commands it can be seen that Maestro allows JavaScript execution. Maestro supports a limited set of built-in functions from a lightweight Vanilla JavaScript.

Here are several ways to execute JavaScript:

1. Inject within ${} blocks:

appId: org.wikimedia.wikipedia
env:
   KEYWORD : User interface
---
- launchApp
- tapOn: "Search Wikipedia"
- inputText: "User"
- scrollUntilVisible:
   element: ${KEYWORD}
   direction: DOWN
- tapOn: ${KEYWORD}
- assertVisible: "Means by which a user interacts .*"

2. Run a JavaScript file using runScript command:

// keywordScript.js
                            
output.keyword = 'User interface'

appId: org.wikimedia.wikipedia
---
- launchApp
- tapOn: "Search Wikipedia"
- inputText: "User"
- runScript: keywordScript.js
- scrollUntilVisible:
  element: ${output.keyword}
  direction: DOWN
- tapOn: ${output.keyword}
- assertVisible: "Means by which a user interacts .*"

3. Use inline with evalScript command:

appId: org.wikimedia.wikipedia
env:
   KEYWORD: USER
---
- launchApp
- tapOn: "Search Wikipedia"
- evalScript: ${output.keyword = KEYWORD.toLowerCase()}
- inputText: ${output.keyword}
- scrollUntilVisible:
   element: "User interface"
   direction: DOWN
- tapOn: "User interface"
- evalScript: ${console.log('The page is opened')}
- assertVisible: "Means by which a user interacts .*"

Moreover, with JavaScript it is possible to interact with API and make HTTP(s) requests.

Supported functions allow to:

1. Pass additional information in headers

// script.js
const response = http.get('https://example.com', {
   headers: {
       Authorization: 'Bearer MyToken'
   }
})

2. Use common request methods GET, POST, PUT and DELETE and more

// script.js
const response = http.request('https://example.com`, {
   method: "GET"
})

3. Send request details in the body parameter

// script.js
const response = http.post('https://example.com/myEndpoint', {
    body: JSON.stringify(
       {
           myField: "Payload"
       }
    )
})

4. Parse JSON responses

// script.js
const response = http.get('https://example.com/jsonEndpoint')
output.script.result = json(response.body).myField.mySubField

Every single feature extends Maestro possibilities and makes this framework more powerful with each new release.

If you prefer to use JavaScript language for test automation then it might be interesting for you to read about Top 5 JavaScript Test Automation Frameworks.

Advantages and Disadvantages

At this point we acquired a better understanding of what Maestro is. Some of the advantages, disadvantages, and limitations of the framework have already started to appear, and definitely more of them will come into view when the framework is used in the project.

Undoubtedly Maestro framework has nice list of advantages:

• Easy to learn and use. Commands are easy to remember and even someone inexperienced in programming languages can start writing test flows right away.
• Active community. Community is small but active, helpful and possesses more information than any available Maestro documentation.
• Community-driven development. Developers listen to the community,  their propositions and reports.
• Cross-platform. Maestro runs on iOS, Android, React Native, Flutter and also Web Views on both iOS and Android.
• Cost-effective. Open-source automation framework that produces good results.
• Integrable with any CI platform. This ability is essential for an efficient development process.
• Tolerance to delays. It helps to decrease the amount of failing steps caused by the unfinished content loading.
• Screen recording and screenshot capturing. Besides the available commands for screen recording and screenshot capturing, Maestro Cloud automatically records flow execution and provides a screenshot of the last application view.
• AI support. Maestro is working towards AI integration, and just implemented it as a new feature.

Yet disadvantages also exist:

• Real iOS devices are not supported. That limits testing possibilities on Windows and Linux OS and might slow down the process on macOS.
• Complex tests can’t be implemented. Maestro supports only a limited set of JavaScript functions that impacts possibilities to create complicated tests.
• Limited execution time. On the Cloud both platforms have a 7 minutes flow execution limit before the timeout will happen.
• Limited reporting capabilities. Reports are generated and stored only in the .xml file.
• Documentation is not complete. As an example, some installation steps are missing and not all command specifications are listed. Have to count on the community.
• Lack of additional resources. Maestro is a quite new framework, you won’t find many guides or analytical data about it.
• Challenging installation on Windows OS. Requires installation of Windows Subsystem for Linux architecture and overall preparation process is time consuming and effort intensive.

When reviewing advantages and disadvantages, it is essential to consider all of the project related specifics and emphasize the things that are important and cannot be omitted.

Conclusion

Summarizing everything, now we can say that we know the basics of the Maestro mobile UI testing framework: how to install it, how to create and run a test flow, what Maestro Studio and Maestro Cloud are, what commands are available and how to execute JavaScript in the tests.

Moreover, we examined some of its advantages and disadvantages.

Maestro is a simple framework but not yet perfect. Yet it is worth keeping an eye on it while the development is still in process. If it’s to your liking, you can give it a try, join the community and help it to grow faster.

If you don’t think Maestro matches your project needs, check a UI automation testing framework called TestUI created by the TestDevLab team! It might be your perfect match!

Read TestUI - UI Test Automation for All Platforms With Appium and Selenide to find out more.