Daily.co: Technical benchmark of Real-Time Video Performance

Daily.co provides a WebRTC-based video calling infrastructure that enables developers to integrate real-time communication capabilities into their applications. To validate platform performance, Daily.co with the help of TestDevLab implemented systematic benchmarks to measure key performance indicators across varying network conditions.

Daily required extensive testing to benchmark their service against competitors. The key areas of focus included:

• Evaluating audio and video quality across fluctuating network conditions (uncapped network conditions, 1Mbps).
• Ensuring stable performance in both 2-user and 4-user call scenarios.
• Identifying any areas of improvement to stay competitive in a fast-evolving market.

TestDevLab’s role in implementing comprehensive audio and video testing was to provide quantitative measurement of audio/video quality metrics, performance analysis under controlled network conditions (uncapped and 1Mbps), scalability testing for multi-participant scenarios, and lastlu, cross-platform compatibility verification

TestDevLab deployed a testing framework that captured objective metrics:

• Audio quality measurement using POLQA scoring
• Video quality assessment through VQTDL and VMAF metrics
• Network performance analysis under variable conditions
• Frame rate and resolution consistency monitoring
• Glass-to-glass latency measurement in a multi-participant scenario

The testing framework provided empirical data across key performance indicators, namely, audio and video performance, and network performance, especially, measured quality metrics in multi-participant scenarios

• Adaptive bitrate behavior analysis: measuring bandwidth utilization patterns and documented adaptation to network constraints
• Quantified recovery times after network events
• Consistent POLQA scores under different bandwidth constraints. Similarly, Frame rate consistency and Resolution scaling characteristics in those different conditions
• Verified performance across different client implementations

Daily.co engaged TestDevLab to perform rigorous benchmarking and analysis of their platform’s performance. We deployed advanced testing methodologies across multiple real-world devices and configurations. Using a wide range of metrics—such as POLQA for audio quality, VQTDL and VMAF for video quality, video resolution and smoothness and frame rate fluidity—our testing process provided a detailed evaluation of how Daily.co’s platform performed under different conditions.

The network performance analysis showed that Daily’s best in class infrastructure and adaptive algorithms have optimal bandwidth utilization patterns under varying network conditions. This is especially observable when the network conditions fluctuate due to user mobility or flaky internet. Daily’s system favors a more conservative outlook on increasing quality too quickly in a very short timeframe, as to avoid fluctuating between poor-great-good, instead favoring a more measured or controlled upgradation.

Through our collaboration, Daily.co gained key insights that enabled them to optimize their platform:

• Top-Tier Audio Quality: Daily.co consistently ranked among the top in audio quality, with excellent POLQA scores even in bandwidth-limited scenarios. This solidified their platform’s strength in delivering high-quality audio during video calls.
• Efficient Network Consumption: The platform demonstrated optimal network consumption rates, allowing for more stable performance in constrained network environments without sacrificing user experience compared to other providers.
• Stable Video Performance: While the video quality was comparable to their competitors, it showed lower variations owing to a more stable ABR. Daily.co also showed stable framerate performance and quick recovery after network limitations were lifted.