As momentum builds in the market transition to 4K UltraHD, the emergence of high dynamic range (HDR) technology is drawing video engineers' attention to new issues that must be resolved if the full potential of a next-generation viewing experience is to be realized.
As various organizations pursue HDR standardization initiatives, the priority must be on choosing solutions from a broad perspective on visual response processes. Along with responsiveness to degrees of contrast, developers must consider several critical human perceptual factors. This will impact everything that's done in the creation and dissemination of video content from initial capture through production, post-production, and processing for distribution.
Light and dark adaptation
One key consideration is the human visual system's ability to adapt to varying light intensity, with key factors including changes in pupil diameter, the impact of luminance on retinal illuminance, the impact on light absorption from the bleaching of photoreceptor photo pigments, speed of adaptation and contrast sensitivity. Settings for HDR must take into account how the human eye adapts to a wide range of content in both bright and dark home environments, and intensely bright outdoor situations as well.
Bleaching adaptation: With the current generation of displays, we are just entering into a point where what is known as "bleaching adaptation" could become significant and impact a viewer's experience. This is the phenomenon that occurs when the eye adjusts to sustained periods of illumination. The bleaching impact will be an important consideration in determining what average and peak levels of HDR brightness should be in the context of temporal shifts in luminance, as the speed at which the eye can receive and interpret light and color will vary when bleaching adaptation is present.
Speed of response and adaptation: According to recent studies, the time course of dark adaptation, the speed at which the human eye adapts to decreased illumination, depends on the intensity and duration of preceding stimuli. If the preceding exposure was low enough that no significant photopigment bleaching occurred, then dark adaptation may also be measured on the time scale of seconds but slightly slower than light adaptation. For bleaching adaptation, dark adaptation may take longer, on the order of tens of seconds to minutes.
Effects of luminance and screen size on flicker perception
For HDTV content viewed on traditional screen sizes, flicker perception - the sensitivity to temporal changes across video frame sequences - has not been an issue. The same is true of judder, the perception of uneven or jerky video playback that arises from movement of objects, edges, or detail from one frame to the next.
Yet with the onset of HDR on large-screen displays, flicker sensitivity and judder could both affect the rate at which content is captured as well as the display frame and refresh rates. This particularly holds true for content captured at the 24 frame-per-second (FPS) rate used with films and episodic series, and could be even more problematic if the motion picture industry raises the capture frame rates to, say, 48 fps.
The effect of luminance on color perception
As luminance increases, so does the ability of the human visual system to discriminate between colors at ever smaller gradations. Consequently, more bits would be needed to code color without introducing noticeable errors, particularly at high luminance. Thus, 10-bit encoding may be expected as a minimum bit depth for HDR for any color space.
Effects of adaptation speed on program changes and commercials
The speed of light and dark adaptation has important implications for the impact of rapid local or global luminance changes in TV programming when HDR and non-HDR content is presented sequentially to viewers. This makes it important for HDR-formatted commercials to utilize the same parameter set for HDR programming. Conversely, non-HDR compliant commercials should not be placed with HDR programming.
The effect of the light field and context on brightness and color
Perceptions of brightness and color are not just a matter of luminance intensity. Variations in the light field across the frame can profoundly influence visual responses as well. Such nuances will come into play as content producers move to using HDR in the creation process. They will need to avoid relying on luminance specifications alone in ensuring their creative intent is conveyed to viewers.
Conclusion
HDR is bringing a welcome transformation in the TV viewing experience to the benefit of producers, distributors and consumers, provided the industry is careful to map parameters to the realities of human perception. Specifically, there needs to be general understanding with respect to the following points:
The requirements of live broadcast programming should be the threshold for setting HDR parameters:
- Dynamic contrast and color ranges will have to deliver an optimal viewing experience without reliance on intervening post-production processes.
- On-site production teams will have to incorporate understanding of HDR to ensure fidelity to creative intent.
The different rates of light and dark adaptation could play a significant role in quality of experience (QoE):
- Light adaptation is fast. Dark adaptation is slower.
- Mixed SDR and HDR could impact QoE for ad insertion, scene changes and program selection.
- Bleaching adaptation could significantly slow adaptation to changes in program luminance.
- Bleach fractions might begin to be significant at 1,000 cd/m.
- They definitely will have an impact at multiple thousands of cd/m.
- It might be beneficial to limit average scene luminance to low bleaching fractions.
- It might be beneficial to limit the duration of high-luminance highlights to minimize after images.
Adaptation could impact user interaction:
- Graphic overlays might need to be tailored to the expected light/dark adaption state.
- GUI could easily be impacted.
HDR should not be considered in isolation. It impacts:
- Color perception with implications for bit depth and color gamut
- Sensitivity to flicker as a function of screen size, frame rate and display refresh rate
Sean T. McCarthy, Ph.D., is a Fellow of Technical Staff at ARRIS.
