Many codecs contains a special parameter ‘VBV size’ (in x264 and x265 VBV-size is specified by ‘-buffer’). VBV-size has a serious impact in case for ultra-low latency applications (e.g. Cloud Gaming). The VBV buffer is a virtual buffer controlled by Rate Control to amortize bitrate fluctuations and not to overwhelm a decoder.
In ultra-low latency applications the VBV buffer size is commonly configured to small sizes (the length is 2x or 3x above the expected frame size).
Because the max-bitrate is measured in the window of 1s, a single large frame can be tolerated without compromising the max-bitrate (if the frame rate is 60fps, we have 60 frames in second, any outlier can be easily compensated by other frames). If the vbv buffer is small the encoder would not produce a large frame which exceeds 2x-3x times the expected size (otherwise the buffer overflow error is inevitable).
Why large-size frames are intolerable in low-latency applications?
Even single large frame (2x-3x times above the expected size) might inflict a spike in end-to-end delay (an user would experience video stuttering). Indeed, a regular frame is transmitted during 16ms (in case of 60fps frame rate), but the frame with the size 3x above the expected would take 48ms to be delivered to the client. Thus, we get the addition of 32ms to end-to-end-delay.
Neither CBR nor CRF restrict the spikes in the frame sizes. In CRF mode the instant bitrate is not restricted at all. In CBR mode the instant bitrate is indeed constrained, but a single large frame can occur without compromising the instant bitrate, since the bitrate is computer over the window of 1s.
Hence, large VBV buffer – higher risk of getting a very large frame without compromising the max-bitrate.
Note: Some codecs provides a special parameter like max_frame_size, which restricts the size of each frame. In such cases VBV size is less relevant.
23+ years’ programming and theoretical experience in the computer science fields such as video compression, media streaming and artificial intelligence (co-author of several papers and patents).
