Standardized Testing Environment for Audio & Video Quality

We test many different audio and video solutions and what we have learned is that every app has its own specifics. Therefore, testing procedures and testing approaches vary from app to app. In order to test such solutions, we need to provide the most simple input feed—audio and video.

To provide audio during the call we use audio cards—also called sound cards—which take digital audio signals and transfer them to analog audio signals. This allows us to input the audio into the mobile device through audio cables (AUX) so that the mobile device thinks that the user is speaking through the microphone.

To provide the mobile device with video media, we can't feed the video straight to the phone and use it as a virtual camera. Instead, we use separate desktop displays to display the video so the mobile device under test can capture the video using its front or back camera.

During these tests, we gather degraded audio and video after the call. The receiving device, also known as the receiver, has received the audio and video from the caller, or the sender. In order to meet quality standards and gather reliable data, we use real devices to simulate real-world network conditions.

But how do we ensure testing processes are accurate, reliable, and provide us with valuable data about audio and video quality? Let’s look at our audio and video quality testing processes in a bit more detail and our standardized testing environment that we have built specially for audio and video quality testing.

Manual testing vs. Automated testing

We do manual testing for most of our audio and video testing activities. The reason for this is because many of the workflows cannot be tested due to certain requirements or experimental specifics. However, for various testing activities like functional testing, repetitive testing and benchmarking, we prefer using automated testing. To automate these workflows we use TestRay, which is a test automation framework developed by TestDevLab. TestRay is a great tool to create and simplify automation pipelines. It allows us to create automation pipelines and test desktop, web, and mobile applications across different platforms. Nevertheless, in order to get reliable results during testing, we need to keep requirements and standards tight.

ViQuLab—Our standardized testing environment for testing audio and video quality

The quality of audio and video solutions can be tested in many different ways. However, the main objective for all of them is the same—test the audio/video solution in the most realistic way possible under simulated conditions.

In order to do that, we use a standardized testing environment called ViQuLab. ViQuLab is a closed testing environment that is isolated from the external environment.

What is inside the ViQuLab varies depending on the type of solution and the type of testing required. Usually, it consists of the sender and receiver device, network and audio handling hardware, desktop display, a main controller device (PC or Mac) which is responsible for running the automation scripts, and audio/video evaluation algorithms. ViQuLab plays a crucial role in audio and video testing because it blocks many different kinds of disturbances that may interfere with the quality of testing and test quality evaluation.

Here is what ViQuLab—our standardized testing environment—looks like:

Additionally, in order to keep the video quality algorithms and the overall testing process false-free, we need to standardize the physical positions of the devices. This factor is important because after each time the tester changes the devices that need to be tested, the physical position changes, which affects the video quality that is being transferred over the call. The change in position of the device also affects the video quality evaluation algorithms which can give false test results, causing a major quality problem.

To avoid such issues, the simplest answer is to develop hardware solutions in order to fix the device positions. These include fixed mountings and stands for the devices under test with fixed X and Z axis, while only allowing the devices to be moved on the Y axis—closer and further away from the desktop display, as different audio/video solutions have different camera resolutions. Keeping this in mind, we can standardize testing procedures according to the specific device under test and the type of audio/video solution.

Apart from eliminating false results, another advantage of ViQuLab is that it can be used not only in test automation but also in manual testing where video disturbances are the most common problem. From our experience, ViQuLab has shown excellent results across different platforms and audio/video solutions.

The bottom line

Having a standardized testing environment for audio and video quality testing allows us to perform a variety of tests across different devices and platforms, while ensuring accurate and objective results every time. Our unique setup is what makes our audio and video quality testing services so popular and trusted by clients.

We follow international standards and use a combination of audio and video quality metrics to thoroughly and efficiently evaluate solutions. We follow the latest industry trends and always look for new ways to improve our audio and video quality testing capabilities. Most recently, we developed VQTDL, our own no-reference algorithm for video quality assessment, which has enhanced the quality of our tests even further.

If you want to learn more about ViQuLab or gain valuable insights into your communications, conferencing, streaming, or on-demand solution—get in touch with us. We can help make sure that your audio or video solution works as intended under different network conditions and meets users’ expectations.