What’s max_bytes_per_pic_denom and how it impacts on frame sizes?

The purpose of the parameter ‘max_bytes_per_pic_denom‘ (which is present in SPS VUI) is to signal to a decoder on the maximal frame size. The decoder can use this info by allocation of input buffer size without a risk to overflow the buffer. If the parameter max_bytes_per_pic_denom  is not present its default value is 2, if it’s zero than maximal frame size is constrained by minCR (minimal compression ratio = uncompresssed_frame_size / compressed_frame_size, e.g. for level 4.0 minCR=4 and for level 4.1 minCR=2 ). The minCR is derived from the level value (see Table A-1 “Level Limits” of H.264/AVC spec.).

Formally the H.264/AVC spec. says:

max_bytes_per_pic_denom indicates a number of bytes not exceeded by the sum of the sizes of the VCL NAL units (in case of slices the picture is composed from a number of NAL units) associated with any coded picture in the coded video sequence. The value of max_bytes_per_pic_denom shall be in the range of 0 to 16, inclusive.

Depending on max_bytes_per_pic_denom the following applies.

– If max_bytes_per_pic_denom is equal to 0, no limits are indicated. [My comment: it’s incorrect, actually the minimal compression ratio minCR]

– Otherwise, no coded picture shall be represented in the coded video sequence by more than the following number of bytes.

( PicSizeInMbs * RawMbBits ) / ( 8 * max_bytes_per_pic_denom )

When the max_bytes_per_pic_denom syntax element is not present, the value of max_bytes_per_pic_denom shall be inferred to be equal to 2.

By the way, x264 (at least the current version) puts max_bytes_per_pic_denom = 0 in the SPS VUI, this means that maximal frame size is constrained by ( PicSizeInMbs * RawMbBits )/minCR.

Let’s take the numeric example, max_bytes_per_pic_denom=1,  PicSizeInMbs =240, RawMbBits =3072 (for 4:2:0, 8bpp) and hence the maximal frame size is 11520 bytes. It’s a violation if a stream would contain a frame exceeding the magnitude 11520 bytes. 

Advice: if your encoder can’t control the size of individual frame than you need configure the encoder to put max_bytes_per_pic_denom = 0. But if your encoder does not send the segment bitstream_restriction (i.e. bitstream_restriction_flag=0) then a decoder would infer  max_bytes_per_pic_denom = 2. Notice that most of decoder ignore max_bytes_per_pic_denom, but there always a stric decoder which would fail decoding your stream without notifying the reason of failure.