Wi-Fi® Device Metrics offers a streamlined methodology to characterize Wi-Fi performance

Wi-Fi^® is no longer just a simple, best effort, wireless link that connects a device to the network for web browsing. While each new generation of Wi-Fi brings exciting new features, use cases are more demanding in terms of throughput and latency, and customers are more concerned by range and robustness. Strong interoperability between devices is not enough for users, they are also looking for very good performance. It is the reason why Wi-Fi Alliance^® has been working to define specific performance methodologies to be tested and quantified.

The need for characterizing Wi-Fi performance

Until now, the main purpose of Wi-Fi certification programs has been ensuring interoperability between Wi-Fi CERTIFIED™ products. Wi-Fi Device Metrics has a slightly different objective: best practices and definitions to standardize performance methodologies of Wi-Fi devices in various use cases. There are many ways to test a given parameter, and a range of performance levels suitable for devices based upon their primary use. Some test cases address the intrinsic device performance, other test cases address user experience.

Orange intends to use Wi-Fi Device Metrics to:

• Provide industry guidelines to characterize and evaluate Wi-Fi devices
• Provide a standard definition of KPIs and performance characterization of Wi-Fi devices

The goal of Wi-Fi Device Metrics is to help operators and IT managers meet the needs and expectations of their users. Wi-Fi Device Metrics encourages that tested devices comply with KPIs defined in the guidelines, encouraging more deterministic Wi-Fi device behavior.

Test cases considered by Wi-Fi Device Metrics

Wi-Fi Device Metrics provides a consistent methodology to evaluate device performances for the following aspects:

• Benchmarking AP throughput, to benchmark the maximum forwarding rate of an access point in both directions with different frame sizes
• Benchmarking AP latency, to evaluate the performance of the AP for delay-sensitive flows under various configurations
• AP rate versus range, to develop a good estimation of the coverage of the AP
• AP channel steering, to checking that the channel selected using the AP automatic channel selection algorithm is always the most relevant, in particular in dense environments
• AP band steering, to check intelligent movement of devices between different frequency bands based on usage, speed, coverage and distance
• Client roaming, to measure the roaming performance based on relevant KPIs such as throughput, packet loss, latency or jitter
  • Moreover, Wi-Fi Device Metrics also considers the specific case of Augmented Reality(AR), Virtual Reality(VR) or Mixed Reality devices

Wi-Fi Device metrics can, for example, help to benchmark performance of AR/VR devices that have high requirements on bandwidth but also – and above all – requirements on latency.

Industry and operator perspective on Wi-Fi Device Metrics

For the industry, Wi-Fi Device Metrics simplifies the evaluation of Wi-Fi equipment by providing a standardized set of tests and related KPIs that characterize Wi-Fi devices so customers can be much more confident in the quality of the product they purchase. The devices that reach the KPIs defined in the test plan will have better behavior, which significantly improves the user experience. Operators can now use the Wi-Fi Device Metrics guidelines for them to select the best products that meets their customers’ needs.

For operators, Wi-Fi Device Metrics significantly streamlines the product selection during RFP processes. Previously, operators had to perform long benchmark tests in order to compare products of various providers during the selection process. These benchmark results and test plans were a source of discussion between the operator and the various suppliers. Since test methods and KPIs were not standardized, they were subject to interpretation and disagreement. Now, using Wi-Fi Device Metrics, operators and Wi-Fi equipment providers can compare their test results, and work together to make products even more efficient. Wi-Fi Device Metrics provides an increased consistency in results across multiple devices.

Moreover, for operators, Wi-Fi Device Metrics also simplifies their validation and customer support processes, making it possible for them to compare data for analysis and troubleshooting. If a customer calls the hotline for a problem related with performance, the hotline has the knowledge of what the performance is supposed to be. For example, in the case where the latency is bad, the hotline will search for a jammer because they know that the problem is not directly linked to the gateway or the device. Test cases defined in the Wi-Fi Device Metrics framework correspond to “the real life” of operators’ customers.

Wi-Fi Device Metrics brings Wi-Fi into a new era: that of efficiency and relationships of trust between suppliers and customers.