How to Set Up Specific Network Conditions for Software Testing?

Network checking is part of the non-functional testing. It is often required for communication applications and other network dependant mobile software to check behaviour over various network profiles, complete loss of network connection and network handover. Since mobile software needs to be designed to be truly “mobile” the testing should check for critical bugs and overall usability of the software being able to cope with the changing network environment.

As we are testing mobile applications every day, we use a number of tools and techniques to test how these applications perform under different network conditions. In this blog post we would like to share our knowledge on how to set up specific network conditions using built-in tools you can find in your web browsers or operating systems. Then we will dive in deeper and show you how we created a more sophisticated solution based on router.

Throttling network with built-in tools

Modern web browsers allow to simulate specific network conditions and it is really simple to set up and use. This is extremely important when building responsive websites, which are meant to perform well on many different devices and in various scenarios.

Google Chrome

Google Chrome desktop browser has a built-in tool for testing under specific network profiles. The network throttling allows to manipulate with upstream/downstream network bandwidth and latency. The tool can be found in Chrome Developer Tools under Networking tab. You can choose from predefined profiles, tweak them or create your own network conditions.

Mozilla Firefox

To access this tool, make sure you’re running an updated version of Nightly, then open the Responsive Design Mode tool via Tools – Web Developer – Responsive Design Mode. You can then select the desired simulated speed with the dropdown menu to the right of “Responsive Design Mode”, and reload the page to see the effect of the changes.

Android (emulator)

If you are developing an Android application, you can limit network speed in Android emulator. You can do it via command line or in Android Studio.

Windows

To create different network profiles on Windows devices, you can use multiple tools and techniques.

• Use Connectify application to create virtual WiFi AP on your PC. It will give you ability to manage traffic over the virtual adapter.
• Download dummynet open source program and install service to your virtual AP.

Now you can connect with another device to your virtual WiFi AP and manage bandwidth, packet loss, latency by using scripts. Here is an example:

Apple (iOS)

You can throttle network on an iOS device with Developer tools. To enable Developer tools connect iOS device to Xcode.

A quick setting of network conditions for other mobile device on iOS:

1. Use iOS device with mobile data;
2. Create hotspot on that device and connect a test device to hotspot;
3. Go to iOS device Settings – Developer – Network link conditioner and enable desired condition.

The condition will be applied to all connected test devices and to the Access Point itself. The network conditioner tool allows to define custom network parameters.

Network conditioning with router

But not all functionality can be tested with tools and techniques mentioned before. There are more complex situations when different approach is needed. We can improve our network conditioning with specially configured router.

The two main use-cases of the router are:

• Isolated way of profiling network for a specific connected device.
• Universal way of test automation that is not platform specific.

Benefits of router:

• Does not affect network for other connected devices.
• Only affects connection to the internet but allows devices to communicate directly in the subnet without interruptions.

Hardware and software requirements

First of all we need to understand what kind of hardware and software is needed to set up our network and what tools we can use for testing in such network.

• We need a router that is compatible with OpenWRT firmware. For this experiment we used TP-Link AC1750 (Archer C7). It should be noted that routers with the same model can have different hardware variations, so please check compatibility guide before starting.
• OpenWRT firmware binary for use with our router. We used Barrier Breaker 14.07 version of OpenWRT because it works well with CDNS.
• Cellular Data Network Simulator (CDNS) for monitoring network usage and simulating network link conditions for mobile applications.

Basic setup and usage

After finding the right router and downloading the necessary software, we can proceed to set up our lab environment.

First of all we need to flash our router with OpenWRT firmware.

1. Enter router management console using web browser (usually defaults to http://192.168.1.1). Open firmware Backup/Restore menu and browse for custom firmware binary image of OpenWRT downloaded to your machine.
2. Follow firmware upgrade instructions and hard reset router after upgrade has completed. You can find detailed instructions here.
3. After firmware upgrade enter the router management console for OpenWRT (usually defaults to http://192.168.1.1) and configure the router defaults:

• set router management credentials – user and password;
• enable WiFi interface and set up secure password;
• enter management tab and enable SSH, change default password for SSH;
• check that the SSID broadcasted is correct and WiFi and internet connections work using the router.

When the router is ready, we can proceed to install CDNS. Connect to the router via SSH and install CDNS following this guide.

The image below shows some situations where two people want to use the same router for limiting network on their test devices. Let’s assume that Alice wants to test some software on her smartphone. And then there is Bob who wants to test something different and use some other test devices without limiting network for Alice. So Alice can pair her smartphone with her laptop and access the router page to control the network of her smartphone. Alice’s laptop and other devices will not be affected and can access network services as usual. In this case Bob can pair the laptop with some other smartphone or a fleet of smartphones that are connected to the same router and make his own group of control that is isolated from Alice’s devices. When Bob completely blocks network for his smartphone then other connected devices remain normal network connection.

Let’s connect a test device (mobile device) to the prepared router and open up administration panel of this router on computer (default IP address is 192.168.1.1). We will see CDNS administration panel instead of OpenWRT dashboard. We can access it via address http://ip-address/index-old.html if we need to change some router settings.

Every device can set its own network profile through CDNS panel. Network connection is set for specific router client by its IP address and others are not affected by that setting. In this case the computer browser can set the network conditions also for a paired test device. In order to pair a test device with the computer:

• Open advanced menu in CDNS via computer browser (Show Advanced)
• Scan QR code with the test device or decode and open the pairing URL in test device’s browser. Once the test device scans and opens the URL for QR code, it is paired with the computer and is listed under Active Devices. Available network profiles can be selected for each active device separately. The specific properties for all profiles are listed in a table under Supported Profiles.

So in the basic usage it is possible to control network connection for a device from another device that it is paired with. Also it is a simple and secure way of sharing the network and router setup with others who can create isolated groups of their own devices.

Advanced setup for test automation

Conditioned router can be used as a building block in test automation setup. Mobile test automation frameworks like Appium, Calabash, Robotium or Selenium use drivers that allow the framework service to communicate with the mobile device and control the Software under test. This means that  the framework also uses the network to interact with the mobile device.

In some cases it is easy to disconnect the test device from the network or limit the connection for it. The consequences are that the automation framework network link is also lost. In our solution the router just limits the communication between the test device and the network gateway. The network link used by the automation framework is not affected and the framework remains in control of the “offline” device.

The router has a RESTful API that allows implementing simple test commands to control the network profiles for specific test devices. In this way the router becomes a great extension for any mobile test automation framework. Read further how to set up and use the REST API of the router for test automation.

REST API with any device without pairing (FIX)

The webapp has RESTful API interface for setting a specific network profile to a selected Client by it’s IP Address. There is a simple hack that allows using REST API and set network profile without pairing.

1. Fix logic that allows setting network profile via REST without paring: Change CDNS set file line 43 to: if [ "$CODE" = "${STATUS[0]}" ] && [ "${PATHS[2]}" = "$REMOTE_ADDR" ]
2. Update the file in the router: scp set root@192.168.1.1:/www/cgi-bin/set
3. Restart CDNS service

root@OpenWrt:/www/cgi-bin# /etc/init.d/cdns stop
root@OpenWrt:/www/cgi-bin# /etc/init.d/cdns start

REST API usage

Router client devices can control specific client’s network via REST API requests. Condition is the network profiles “Friendly name”.

Set network condition via REST

<router_home_IP>/cgi-bin/set/<Condition>/<ConditionedDeviceIPAddress

Example:

GET http://192.168.1.1/cgi-bin/set/full/192.168.1.126

Router will respond with 200 OK if condition is set. Following is a JavaScript sample implementation for a command that sets a specific network condition and checks if it is set.

const Promise = require('bluebird');
const request = Promise.promisify(require('request'));
module.exports = {
  interpret: function (routerUrl, pcUrl) {
    const condition = this.params.condition;
    const options = {
      method: 'GET',
      url: `http://${routerUrl}/cgi-bin/set/${condition}/${pcUrl}`
    };
    console.log(options.url);
    this.addMessageData('request', options);
    return request(options).then(response => {
      this.addMessageData('response', response);
      if (response.statusCode !== 200) {
        throw new Error('Failed to set condition');
      }
      this.result = {
        status: true,
        string: response.body
      };
    });
  },
  getCommonsParams: function () {
    return ['@routerUrl', '@pcUrl'];
  }
};

Custom network profiles

CDNS service uses Linux packages TC (Traffic Control) and Netem for directly setting specific network parameters between network interfaces in a way that adds minimal interference and is applied in a fast manner. Default CDNS package is shipped with a smaller set of mobile network profiles:

• Full (no restrictions)
• HDSPA (3G+)
• EDGE (2G)
• GPRS(1G)
• Disabled (100 % packet loss)

The network profile configurations are stored in a config file that is loaded once CDNS service starts on the router. Each profile (network type is characterised by a set of network metrics). The metrics can be set for inbound and outbound communication directions and should note that they sum up for bi-directional communication. The main parameters that can be configured are:

Adding custom profiles

a) Edit default profile configuration file. Add new network profiles and specify the profile name as well as the options. Example profiles for most common scenarios are added to the file.

b) Replace and store the profile configuration file in the router. Restart the CDNS service.

#Replace the file in the router

scp cdns root@192.168.1.1:/etc/config/cdns

#Restart CDNS service 

root@OpenWrt:/www/cgi-bin# /etc/init.d/cdns stop
root@OpenWrt:/www/cgi-bin# /etc/init.d/cdns start

c) Open CDNS web app and check under Advanced menu that the profiles are loaded and displayed.

Known issues and troubleshooting

• The CDNS webapp has a known bug with improper reloading in browser. It can be reloaded without caching.
• The webapp does not load. Check that CDNS service is started and running.
• Specific network condition is actually not set. (Can be checked with ping tool after condition is set).

Fix for netem :

a) Update packages via ssh

opkg update
opkg install tc iptables-mod-ipopt kmod-sched kmod-netem

b) Reload netem module

insmod sch_netem

There are many ways of network profiling. Most of the development platforms have created their own tools for testing under network limitations. Yet the ways of testing cross-platform and automating specific network types and conditions require more sophisticated setup, therefore, here is just one device that works as a universal tool for network fiddling.