Enhancement of Error Resilience by Non-reference P Frames

 

Content

Idea

How much non-Reference P-framing impacts on the video quality?

Nvidia Rate Control has a flaw

 

Idea

For low-latency applications like Cloud Gaming, in case of packet losses a retransmission is not feasible. Consequently visual distortions are observed for a while, due to temporal prediction, if the previous frame is corrupted then the current frame is also corrupted (since each frame uses the previous as a reference).  In case of the packet loss event Client is expected to notify immediately Server and Server in turn should immediately send IDR (or intra-refresh) to clean up visual distortions. Because a round-trip of 100ms or greater the visual distortions are present and magnified.

 

Traditional low-latency GOP structure, due to the low-latency B-frames are not utilized:

Here the red-colored frames denote the corrupted

 

Non-reference P-frame GOP structure (each second P-frame is non-reference):

Non-reference P-frames lessens visual distortions in case of packet losses, generally speaking 50% probability that visual distortions due to  a bitstream error or a packet loss is confined in a single frame.

However, if you need better error resilience you have to sacrifice the video quality – some degradation of coding efficiency (and hence the visual quality) is expected because the half of P-frames refer to non-immediate reference.  

 

 

 

 

How much non-Reference P-framing impacts on the video quality?

Testing Methodology

• Codec:  H264/AVC
• Device: Tesla T4,  Encoder: Nvidia SDK10
• Target Bitrates: 10Mbps, 20Mbps
• Encoder Configuration: preset p3, dual pass VBR Rate Control, gop length 512, gop type IPPP (low latency):

AppEncD3D11.exe -i <yuv-file> -s 1920x1080 -codec h264 -profile high -gop 512 -bf 0 -preset p3 -rc vbr -initqp 28,36,22 -fps 60 -bitrate [10Mbps or 20Mbps] -tuninginfo hq -multipass 1 -o <h264-file>

 

• Video Quality metric: VMAF (average, 5%-percentile and min. scores)

Note: Due to instability of Rate Control at the start, the first 20 frames are excluded from consideration.

 

• Video sources 1080p@60fps

Non-reference denotes the mode where each second P frame is not used for reference (in order to enhance error resilience)

FIFA

 

 

SWBF

 

 

Unravel

 

Nvidia Rate Control has a flaw

If P-frame is not used for reference its bit-budget reduced. This causes fluctuations in  both vmaf and psnr scores which can be noticeable.
Picture sizes look a comb-like structure, each second (i.e. non-reference) frame has smaller size:

Fluctuations in picture sizes is reflected in fluctuations of quality scores:

Segment of PSNR scores

Legend: green is regular mode, brown is non-reference

 

Segment of VMAF scores: