Rate Control Et Cetera

Rate Control Issues:

There are two main types of Rate Controls: CBR (Constant-Bitrate) and VBR (Variable-Bitrate). In addition some encoders support CVQR (Constant Video Quality Rate Control), CVQR tries to keep constant visual quality. E.g. x264/x265 has a particular command-line parameter ‘-crf’ to specify CVQR mode and to keep a selected “visual quality”.

A video sequence typically has time varying complexity (still scenes are followed by fast moving ones and vice versa). Therefore in order to keep a constant visual quality it’s required to enable fluctuations of instant video bitrates. In the constant bit rate (CBR) mode bitrate fluctuations are restricted and time-varying visual quality degradation can be observed.

If a latency is not must in your application, use a lookahead mode (and even dual pass mode) to improve Rate Control decisions, the encoder looks a number of frames and collects statistics of theirs frames to make mode decisions more adequate.

Decisions of Low-latency Rate Control (where lookahead is not possible) are based on long-term history (at least from the recent scene change).

Most of Rate Controls are based on the following assumptions:

1) Prediction residues are distributed as Gaussian or Laplacian

2) Rate-Distortion function D(R) is power function model, e.g. for low bitrates (R < 1bit/pixel) D(R) can be approximated as D(R) = C/R.

There are two main classes of Rate Controls:

Virtual-buffer based (implemented in x264 as well as in x265). Such kind of Rate Control is based on virtual buffer model. The encoder tries to not violate (usually not to overflow) the virtual buffer. The virtual buffer parameters:
- size, often called as ‘VBV_size’
- draining bitrate, often called ‘VBV_bitrate’
- initial virtual buffer fullness before starting drained

The idea of virtual buffer appeared a while ago when video was transmitted via telephone or cable TV networks and constant bitrate was must.

From the figure above output of Video Encoder is sent into a smoothing buffer that is drained in a constant rate. In order to keep the constant drained rate the smoothing buffer must not overflow or underflow. The feedback controls Video Encoder output rate to not violate the smoothing buffer. Rate control controls fullness of the smoothing buffer by adjustment of quantization factors, rarely by frame rate.

ρ-domain rate controls (unfortunately not realized in x264). The relationship between rate and quantization is built through the special parameter ρ, which
is the percentage of zeros among the quantized transform coefficients.
ρ-domain rate control is found to be skewed for low bitrates since significant amount of bits are spent on MB/CTU overhead (motion data etc), therefore an impact of residential bits gets weaker.

In non-multicast case Encoder can get a feedback from Receiver on massive packet losses and in such case Rate Control adjust quantization parameters to reduce output bitrate.

Improvement of Rate Control can be achieved if Content Analysis added:

The above figure is taken from the paper “Rate Control Technology for Next Generation Video Coding Overview and Future Perspective”, Hao Zeng et al.

Note:

Most of Rate-Control implementations are based on R-Q relationship (Q stands for the quantization step) as an approximation of R-D curve. How accurate is R-Q for low-bitrate scenarios? In high and moderate bitrates the lion’s portion of bits belong to residual values. The residual values are directly impacted by Q. In low-bitrate cases a significant portion of bits belong to CTU/MB overhead (e.g. motion data), R-Q relationship is expected to be less accurate. Changing Q might lead to a negligible change of the bit-size of a CTU/MB.

Use Case of Frame Level Rate Control

Rate Control of x265

Chicken and Egg Dilemma in Rate Control Implementation

Is it correct to double the bandwidth when frame rate of video switched from 30Hz to 60Hz?

23+ years’ programming and theoretical experience in the computer science fields such as video compression, media streaming and artificial intelligence (co-author of several papers and patents).

the author is looking for new job, my resume