Start Code Emulation Prevention in H264 and HEVC

AVC/H.264 and HEVC/H.265 elementary video streams contain a particular bit-patterns (called start codes) 0x000001, these patterns are dedicated to delimit NALUs (e.g. frames or slices).

The problem is that both AVC/H.264 and HEVC/H.265 can’t guarantee prevention of emulation of the start code within NAL data. Let’s imagine a decoder looks for the start code to enter to the stream and to commence decoding and is locked on an emulated start code? In such case the decoder might crash. 

To prevent start code emulations an encoder has to insert so called “emulation prevention byte” 0x03, in other words Encoder has to replace each emulated start code 0x000001 (within NAL data) with 0x00000301. Formally this post-processing operation is called the conversion from RBSP (Raw Byte Sequence Payload) into SODB (String of Data Bits). 

Obviously, as a preprocessing step a decoder in turn has as to remove emulation prevention bytes (i.e. to replace 0x00000301 with 0x000001), this operation is called the conversion from SODB into RBSP.

Elementary streams are delivered to clients via containers (e.g. MPEG2 Transport Stream) and the start code can be emulated within headers of a chosen container.

For example, MPEG2 Transport Stream format has a particular treatment to avoid start code emulations:

• In PES header – the marker bit (always equal to 1) is inserted in PTS/DTS fields (as well as in ES_rate field) to avoid any chance of the start code emulation.
  
• Stuffing bytes in transport packets are required to be 0xFF (not noise data where the start code can happen in random).

However, on the other hand, MPEG System does not constrain the content of null packets – data_bytes may be assigned any value.

“Any value” can contain also emulated start codes. It’s a good style for transport stream multiplexor to fill up null packets with data equal to 0xFF (to avoid start code emulation), but it’s not mandatory. In some sense non-restricting content of null packets is a flaw of MPEG System.

There is more one place where start-codes could be emulated – private data fields. Fortunately, the MPEG2 forbids the start code (0x000001) emulation in the ts-header private data.  

The MPEG-2 spec. says:

PES_private_data – This is a 16-byte field which contains private data. This data, combined with the fields before and after, shall not emulate the packet_start_code_prefix (0x000001)

PES header is also prefixed by same start code 0x000001 and PES header contains marker bits to avoid potential start code emulation. Notice that a start code emulation might cause problems in Middle Boxes.