H.264/AVC Error Handling

Content

Prelude

Notes on Random Access Points

Error Detection Latency

Error Recovery and Concealment

Slice Data Errors Recovery and Concealment

NAL Header Error Recovery & Concealment

Lost Field Handling

Details of Data Concealment

Additional Recommendations To Enhance Error Resilience in AVC/H.264 Streams

Mathematical Expectation of Number Corrupted Frames in Packet Loss Channel

 

Prelude

Packet losses and bit-flips occur in video transmissions. Losses can have a very destructive effect on the video quality. If the system (actually a decoder) is not designed to handle losses, then even a single bit error can have a catastrophic effect. Wired packet networks such as the Internet are mainly afflicted by packet loss, where a congestion in an intermediate node may cause an entire packet to be discarded.

On the other hand, wireless channels (as well as satellite communication due to the solar wind) are typically afflicted by bit errors or burst errors at the physical layer.   So, bitsream errors (e.g. bit-flip) and packet losses are inevitable.

There are four main approaches for error control: 

1. Retransmissions (not tailored for low latency, back-channel is required). In retransmission-based approaches the receiver utilizes a back-channel to notify the sender on lost packets and the sender it turn retransmits the lost packets.

2. Forward error correction (FEC), when data is transmitted via a wireless channel where a single bit-flip can occur with a reasonable probability. In FEC-based approaches a specialized redundancy data is added (at the end or at the start of a packet) to assist to recover (or to correct) data from a few bit-flip errors. Compared to retransmissions, FEC does not require a back-channel and suits low delay applications.

3. Error concealment – the simplest approach: if a frame is corrupted don’t display this frame and ask a clean-up from Server (in H.264 or HEVC either IDR-frame or intra-refresh is requested from the Server).

4. Error-resilient video coding, e.g. scalable multi-layered video, data partitioning, reverse VLC (this mode is present only in MPEG-4).

 

How Decoder detects bitstream errors?

• At the network level: sometimes packets contain special hash values (e.g. CRC32) and if a bitstream error occurred in the packet then the hash value is inconsistent and this packet is considered as dirty.
• At the elementary stream level (relevant for H264 and HEVC in AnnexB format):
  • if during the decoding process the decoder sees the start code while MB/CTU is not finished yet, then it indicates that a bit-stream error has occurred and the current frame (or slice) is invalidated.
    
  • if during the decoding process the decoder has finished a frame but no start code is not present, then it indicates that a bit-stream error has occurred.
  • The dissimilarity metrics are employed (for reference pls. look at the paper “A Hybrid Error Control and Artifact Detection Mechanism for Robust Decoding of H.264/AVC Video Sequences“, by Reuben A. Farrugia et al., 2010):  the average inter-sample difference across MB boundaries, texture consistency etc.