LiveQ: End-2-End Audio & Video Testing with Live Reporting

Audio & video quality testing is crucial for any media-sharing application. Whether it be a single service app, such as a streaming or video conferencing app, or a multi-feature social media app, it is really important to be aware of the quality of the media that reaches your audience. As the fierceness in competition between platforms increases, providing poor or slightly less acceptable quality than competitors may push away the user base.

At TestDevLab there has been a lot of effort put towards developing cutting-edge technologies to give the most comprehensible quality measurement for these applications. For example, we have developed algorithms to obtain new metrics and KPIs (VQTDL, Frames per Second, Audio MOS), and designed laboratories specifically suited for this kind of testing.

However, up until now we have been relying on post-processing pipelines, where we first did the testing, recorded all the metrics, and processed all the media files to get the final results. This is both time-consuming and resource intensive, so we came up with the new Live Quality Assessment pipeline or LiveQ. This pipeline takes the media, such as audio, video, network traces, or similar, and processes them on the fly. So let's dive into this process and see how it works!

LiveQ Architecture

LiveQ is a set of software that can be used to capture any sort of metric and upload it to a database; these metrics can then be parsed by an API and displayed in a web interface, called Tempero LiveQ, for data analysis. We had already developed scripts to measure media quality, but now they have been adapted to be able to capture live metrics from:

• HDMI Video Streams (forwarding mobile video feeds) and Computer Screens, where we can get the frames per second, the picture quality, and other characteristics.
• Audio, by running Analog / Digital audio signals through audio cards. These audio signals can be analyzed with algorithms such as POLQA, ViSQOL, and two of our in-house developed metrics: SpeechQ (Measuring the quality of the speech transcript) and ASQ-ViT (Audio Spectrogram Quality with Visual Transformer) which is a non-reference audio quality algorithm.
• Network Traces, give us information about what the network consumption is, where the traffic comes from, and which network protocols, as well as other data, are embedded within the recorded packets.
• Other device performance metrics, such as RAM or CPU usage.

While the media is being captured, it is simultaneously being analyzed and sent to a database; this applies to all captured metrics available for analysis, efficiently executed in parallel threads. Moreover, the raw media files are stored to be later used for correlating the media with the quality metrics.

What to Do with Live Quality Metrics?

Once we have stored all the data, we can retrieve it by using the API from the Tempero LiveQ portal. We can use this data to check its behavior over time, understand the effect of new releases on the media quality, and check how network impediments affect the quality, while simultaneously giving us the ability to establish thresholds of acceptance. For example, an application should not deliver a MOS score lower than 3.0 in any of the given network scenarios.

We can emulate network conditions in our laboratories, so the scenarios closely resemble real-life situations, such as those featuring low bandwidth, high packet loss, jitter, or delay. This gives us and our clients better insight into the way the application handles different scenarios, and how they perform against competitor applications.

It is possible to add these results to the internal pipelines and raise alerts or failures in case something unexpected occurs. This enables the integration of audio & video automation within the pipelines with minimal testing time.

Setting up your Audio & Video Testing

If, as of today, you don’t have any quality assessment over the media your application shares, we can handle that for you. We can set up the pipelines, build appropriate labs and even create the reports for you. With this set of software and laboratories, we can build a customized solution tailored to your needs. If you already have the media streams and you are just looking to obtain the metrics, we can also fine-tune our algorithms to grab the media, analyze it and display the results in a concise manner that is relevant to your needs.

We have developed this software to be able to test any form of video and audio, and have specialized laboratories to test desktop, Android, and iOS devices. This way you can check the quality for any single platform, or platform combination. With our device coverage, spanning over 3500 real devices, available for immediate testing, we can provide the best competitive analysis.

If you want to get more information about our solutions or any other testing services, contact us and we will make sure that your testing needs are met and you stay on top of the competition!