• Subscribe
  • Videos
  • White Papers
  • Webcasts
  • Diamond Tech Reviews
  • On Topic
  • Press Releases
    1. Test
    2. Test & Measurement

    Proving network performance

    June 12, 2019
    Thanks to the growing importance of QoE, a growing number of service providers are combining traditional QoS testing and newer QoE test methodologies like RFC6349 and internet speed testing methods.
    Binary 1607161 1920
    Figure 3. 1GbE data rate versus BDM.
    Figure 3. 1GbE data rate versus BDM.
    Quality of experience (QoE) is the most critical metric in achieving consumer satisfaction and reduced truck rolls. Unfortunately, tech savvy customers often attempt to validate their service level agreements (SLAs) with off-the-shelf equipment, including laptops, tablets or smartphones. Such CPU-based platforms don’t have the capabilities required to reliably test gigabit broadband services at full line rate, much less today’s 10G networks. In their disappointment, customers mistakenly blame the network instead of realizing that their hardware is the bottleneck.

    Such dissatisfaction causes trouble tickets, truck rolls and high customer attrition. The problem is further compounded when technicians use similar off-the-shelf hardware and software to validate network performance.

    Service turn up and testing are important first steps to avoid unnecessary operational expenses after the service is delivered. Also important is quality of service (QoS) testing, accomplished via traditional Layer 2 and Layer 3 test methodologies like RFC2544 and ITU-T Y.1564. But today, thanks to the growing importance of QoE, a growing number of service providers are combining traditional QoS testing and newer QoE test methodologies like RFC6349 and internet speed testing methods.

    1G and 10G: Proving both is a challenge

    The future of the 10G platform is now becoming reality as broadband providers who have already proven consistent gigabit performance are reaching to meet consumer demand for faster speeds. At 10 Gbps, service providers lay the foundation upon which consumers will interact with the digital world every day.

    The deployment of 10G networks also brings new challenges. These networks overwhelm available off-the-shelf hardware even more than 1G does, which means the hardware is unable to achieve the full 10G line rate. This factor leaves the service provider unable to verify 10G network performance.

    And the problem is even worse with the do-it-yourself approach. A problem is bound to occur when the consumer tests their service with an older laptop that has a slower CPU and is unable to achieve 1 Gbps over its copper RJ-45 10/100/1000Base-T interface. Aggravated consumers that are unable to validate their 1G networks can become a nightmare for service providers when the service goes beyond 1 Gbps, as there is really no consumer laptop available today with a 10-Gbps interface. For example, the consumer can pay for a 2-Gbps DOCSIS 3.1 service but is unable to test the network speed with their top-of-the-line video gaming laptop.

    Once they realize that their laptop is only measuring 200-300 Mbps out of a 2-Gbps service, they can become frustrated and may file an unwarranted complaint with their service provider. This complaint in turn will generate a ticket that could turn into a truck roll, an unnecessary operational expense for the service provider.

    To eliminate deployment failure and remove any doubt that the broadband service meets the SLA, service providers should invest in dedicated test equipment that exceeds any consumer device performance to carry out stateful TCP testing. Consistent and relevant testing methodologies, as well as dedicated and specialized hardware, are needed for testing QoE and meeting SLAs with consumers.

    For this reason, leading service providers are complementing traditional RFC2544 and ITU-T Y.1564 testing with QoE testing. These service providers use the RFC6349 methodology and simple upload/download testing to ftp/http servers to validate network performance. This stateful TCP/IP (Layer 4) testing builds consumer confidence in the provider.

    Both internet speed testing and the RFC6349 testing strategies will require the deployment of dedicated TCP servers within the service provider footprint or dedicated rackmount test equipment. This dedicated equipment should be able to operate as both a TCP server and TCP client when testing against field equipment, in addition to carrying out other traditional Ethernet test methodologies like ITU-T Y.1564 and RFC2544 (see Figure 1).

    Figure 1. Broadband network with deployed test heads.
    Figure 1. Broadband network with deployed test heads.
    Meanwhile, field service technicians and engineers require dedicated and specialized test tools beyond what is standard issue from service providers (conventional tablets, smartphones and laptops) to successfully deliver an accurate proof-of-service for their customers. Using dedicated tools designed for specific tasks helps provide repeatability and reliability in the testing methodology and procedures. This is important during the installation and delivery of new services. In addition, these tools provide the required physical interfaces to test the services: 10/100/1000Base-T, 1000Base-X, 10GBase-X and DOCSIS 3.0 and 3.1.

    Testing performance

    A controlled test between a client connected directly to a TCP and HTTP server proved that when a 1-msec delay was introduced (for example, due to a network impairment), throughput dropped by less than 10% when using dedicated hardware. However, throughput dropped by more than 50% when using off-the-shelf laptops. A 10-msec delay between a client and a server remain as expected (see Figure 2).

    Figure 2. TCP throughput versus delay.
    Figure 2. TCP throughput versus delay.
    Similar results were found when testing Layer 4-7 both in the field and in a controlled lab environment. There is a clear need for dedicated test tools for TCP-based applications when testing 1-Gbps broadband services and beyond. Most laptops and tablets with software-based clients can only reach a certain level of reliable throughput performance. None of them can reliably verify SLAs for broadband services at or beyond 1 Gbps as well as dedicated hardware/FPGA-based instruments do. Service providers need to take this into account as they enter the gigabit services market.

    As Figure 3 shows, even if dedicated hardware is used on the client side but not on the server side (white line), it still does not match the performance of dedicated (blue line) hardware for both client and server or calculated ideal (green dotted line).

    Figure 3. 1GbE data rate versus BDM.
    Figure 3. 1GbE data rate versus BDM.
    Technologies deployed for service assurance vary and may include FTTx, DOCSIS 3.0/3.1, GPON, point-to-point Ethernet or XGS-PON. Regardless of the technology used and deployment strategy, all service providers face the challenge of proving that their broadband service is being delivered to their customers as promised. To meet this challenge, service providers need reliable and repeatable test methodologies delivered via the proper devices to adequately support their service efforts.

    Ricardo Torres is director of product marketing and a founding member of VeEX Inc. He leads the product marketing, strategic positioning and product management of VeEX’s Ethernet portfolio. He is also responsible for the company’s global business initiatives encompassing Carrier Ethernet/IP networks, mobile backhaul, 40GbE/100GbE and Ethernet synchronization technologies. Prior to joining VeEX, Ricardo worked at Sunrise Telecom, where he was responsible for managing the Ethernet/IP portfolio. He has also worked at Agilent Technologies in the area of high-speed fiber-optic networking.