ABR test strategies from RF through content delivery

May 1, 2017
As the trend toward over-the-top (OTT) services continues to grow among video service providers and cable operators, so does the demand for a high quality of experience (QoE) on any device and at any time. But delivering the quality viewers expect involves the mastery of very complex adaptive bit rate (ABR) technology and implementation of a comprehensive set of monitoring capabilities and tools.

As the trend toward over-the-top (OTT) services continues to grow among video service providers and cable operators, so does the demand for a high quality of experience (QoE) on any device and at any time. But delivering the quality viewers expect involves the mastery of very complex adaptive bit rate (ABR) technology and implementation of a comprehensive set of monitoring capabilities and tools.

With ABR going mainstream, you now need to not only ensure that the transcoders are creating the appropriate profiles, but that the video is being monitored from acquisition (e.g., satellite, fiber, file) all the way to the origin server and content delivery network (CDN) servers. If you want the ability to proactively detect and repair errors quickly, you’ll need taps or test points at each of three critical points in your ABR system as shown in the illustration below.

Figure 1. The three critical test points in a typical ABR system.

Test Point 1: RF Demodulator

One reason to test here is that transmission errors are easy to detect and quantify. These errors will become hidden or much harder to trace back to their origin once the video and audio elements have been unwrapped or de-encapsulated from their previous transmission chain.

For a sense of what can happen during the transmission, look at the results in the screen capture below taken from a satellite signal over a one-hour period. As you can see, it shows a variety of Digital Video Broadcasting (DVB) section errors as well as program clock reference (PCR) interval and continuity counter errors. The continuity count (CC) errors are considered extremely bad since they denote a missing packet, and the decoder will struggle or fail during the decode process. PCR errors are not good either, but treated at a lower priority due to the filtering functions built into each set-top box or TV receiver.

Figure 2. Typical errors that can occur in a satellite signal.

Another aspect to ingest or content acquisition is file-based. This is where the entire program, event, or movie has been software encoded in non-real-time and delivered as a large file. At Test Point 1, these source files are usually encoded at very high rates and are not for subscriber viewing. A popular format for these files is interoperable master format (IMF). This format will include a composition play list (CPL) that includes reference to its video, audio languages, captioning, and ancillary tracks.

Each ingested file will be used as a reference for the transcoder to create multiple profiles that can be easily transmitted (lower bit rates, higher compression). Before each reference file goes to the transcoder, each file should be tested for compliance to ensure that it is interoperable with any downstream decoding device. It is also prudent to check for blockiness, for frozen frames, and that the audio loudness levels are within specification and therefore don’t violate regulations. Once tested, files can be moved to another folder or drive for transcoding, or off to a quarantine folder for files that fail the compliance or quality tests.

Test Point 2: After the Transcoder

The next step in an ABR system is to transcode the program from its incoming format to a newer codec format such as H.264 at a higher compression rate. For both live events and file-based content, each input stream or file will result in an array of many outputs or profiles for different bit rates. Each of the new streams (profiles) should be verified for its rate, format, syntax and semantics, blockiness, and loudness, as well as making sure the new reference frames are time-aligned across each of the profiles. These new reference frames (instantaneous decoding refresh [IDR] and encoder boundary points [EBP]) should occur at predetermined intervals. These intervals or boundaries are the points at which the ABR profiles can be switched while continuing to provide a seamless viewing experience.

One of the biggest problems with video compression is blockiness. This issue is due to the encoder not having enough bandwidth to maintain quality. A key problem here is that every encoder will react differently to the same content. The same set of encoders with different levels of firmware will react differently too.

A common issue is that the video frames look clean at any rate without any blockiness if the content does not include fast action scenes. In the zoomed-in example below, the image on the left is from a 50-Mbps/MPEG-2 source file, while the right image shows the blockiness in the hand and arm area that can occur when there isn’t enough bandwidth to support all the movement in the video.

Figure 3. Blockiness from insufficient bandwidth to support the action in the video seen in the image on the right.

With all video, there will be scenes with little movement that are easy to compress, while other scenes will include fast action scenes that are harder to compress. Finding the correct balance with all of the different codec settings is the challenge. To be sure you’re mapping as close to the original as possible, it’s best practice to check for blockiness each time a video is encoded or transcoded.

At the heart of any ABR system is the packager. The packager performs three main functions. It transcodes the input content into multiple bitrates (on many systems there can be eight different bitrate profiles produced). Next, the packager fragments the profiles into 2- to 10-second blocks. Finally, it encrypts the fragmented profiles for delivery to the origin or CDN servers. In some cases, the transcoded output is accessible for testing, and in other cases it is only accessible after fragmenting and encryption.

In the case of a server using digital rights management (DRM) software, it is now possible to test the manifest, profiles, and the video/audio quality using DRM within the monitoring equipment. This capability enables the test equipment to view the same content that a subscriber will see.

Test Point 3: Origin and CDN Servers

Once the TV and video programs have been placed onto the servers for playout, you can now monitor the manifest files that describe the multiple profiles for each program. You can also check the latency and bandwidth of the requests, and then analyze the viewer’s experience by decrypting the content. In most cases, it would not be wise to use a single server (i.e., origin server) for all subscribers.

For large systems, it is important to push the content as close to the subscriber as possible. This means making multiple copies of the most popular programs and storing them at caching sites around the region. Testing all of the content at all of the sites throughout every region would be great, but most service providers tend to focus on the content at the origin server.

The first test at this point should be to verify that the manifest and associated profiles agree with each other. It is critical that the monitoring system sends an alert when it detects requested fragments taking longer to arrive than their allocated runtime. For example, it should never take more than 10 seconds to download content that only plays for 10 seconds.

When testing at the origin or caching servers, you may want to test everything, but you do not want to create a scenario where the monitoring equipment requests more content than the network will allow. Also, you do not want to consume so much bandwidth that subscribers can’t download and watch their requested programs. Therefore, the monitoring equipment should be configured with maximum thresholds that limit the amount of content being pulled from each server.

To complete the end-to-end quality of experience for ABR content, the monitoring equipment should decrypt and decode the live pictures and audio from the origin or caching servers. In most, but not all cases, the content is encrypted. With DRM support, monitoring equipment can decrypt each program and then look at the same video and audio available to subscribers. The bandwidth for each representation should be very stable and consistent over time. The quality of the decoded video and audio should be good as well.

The snapshot below of a manifest file shows a video thumbnail and its program quality rating. These features are useful for troubleshooting and diagnostics, as well as long-term monitoring with triggered alerts.

Figure 4. This representation of a manifest file shows measurements for quality of experience and blockiness.

Get monitoring

With ABR streaming services, it is important to monitor content from ingest through transcode, fragmentation, and encryption for all available profiles. It is critical to establish and test content at the beginning of the chain to ensure what is going into the system is a known good entity and follow it through the transcode process. On the delivery side, providers need to test that the content will be available at various bit levels when requested and that it will play successfully. Ultimately, it is about ensuring that what goes in is great and what comes out leads to the best possible experience for your viewers.

Dennis Kucera is an application engineer at Tektronix, where he has held technical positions for over 30 years. Kucera is the inventor of the oscilloscope Color Graded Display and Automated Mask Testing, and is credited with the U.S. patent for Automatic Eye-Diagram Testing. In 2007, he received (on behalf of Tektronix) a Technical Emmy from NATAS for Monitoring MPEG Broadcast Signals. He has published multiple industry articles on video design and testing. He holds degrees in Computer Science and Mathematics from the University of Oregon.