Video Compression

VideoNerd

Introduction

The purpose of the list reordering feature of H264/AVC is to show how reference frame reordering impacts on  coding efficiency.

This document describes in details how to make the reference list reordering in order to achieve better coding efficiency. The H.264/AVC supports special reference list reordering commands (signaled in the slice header) and which are applied on both reference lists: L0  (previous frames) and L1 (future B-frames) . The same reordering commands are performed in Encoder and Decoder, otherwise a drift (or async) between Encoder and Decoder is inevitable.

 Note: This shuffling by reordering commands is temporary and is valid only within a given frame (more precisely within a given slice).

The reference picture indexes  ref_idx_l0 and/or ref_idx_l1 (if B frames are used) is signaled for each MB inter partition, excepting the skip or the direct modes (in both skip and direct modes ref_idx is derived from previous values of ref_idx).

The parameter  ref_idx_l0[l1] is binarized as a truncated  Exp-Golomb code and then forwarded to Arithmetic Engine. Large values ref_idx_l0 or ref_idx_l1 require much more bins than low values. Indeed,  if the reference lists consists of  four references: Ref0 (1 bin), Ref1 (3 bins),  Ref2 (3 bins) and Ref3(5bins). 

When most the distant reference is more beneficial than the immediate previous one. The  situation of flash lights  is not uncommon, especially in gaming content. In cases of flash lights the reference Ref3 is a good candidate for reference, despite its distance to the current frame, since the previous frames are “corrupted” by flash lights:

 

The problem is that Ref3 requires 5 bins while Refo 1 bin. Hence Ref3 index is bits-consuming. However, if we reorder the reference list for all slices within the current frame such and to place Ref3 at the first position (with index 0 and 1 bin) and Ref0 is placed in the last position respectively (the index 3 with 5 bins). In such case a gain in coding efficiency is expected. So, it’s worth to reorder the reference list as follows:

    Ref3, Ref1, Ref2, Ref0

Notice that reordering commands are signaled in the slice header, hence regions “corrupted” by flash lights are reordered.

 

 

Reference List Reordering Syntax

Reordering is signaled in the slice header with the following syntax elements (here the suffix ‘x’ denotes the refrence list L0 or L1)  :

ref_pic_list_modification_flag_lx –  if 1 then reordering commands are present in the slice header and are conducted by both Encoder and Decoder in the same way (to avoid a drift between Encoder and Decoder).

abs_diff_pic_num_minus1 

modification_of_pic_nums_idc  – together with abs_diff_pic_num_minus1  indicates which of the pictures in the reference list is moved. The value 3 means termination process.

 

 

Algorithm of Shuffling Reference Pictures

A tacit assumption, a frame_nums of all pictures in reference list are less than 2log2_max_frame_num

 

 // set initial prediction values

if field_pic_flag==0    // i.e. the current picture is frame and not a field

                       picNumPred = frame_num of current frame

          Else  // fields, interlace video

                   picNumPred = [frame_num of current frame]*2+1

 

          RefIdxLX=0

Read modification_of_pic_nums_idc   // read first reordering command

while (modification_of_pic_nums_idc!=3) // ‘3’ stands for termination

           {

                   Read abs_diff_pic_num_minus1

        // compute frame_num of  frame to be moved

         picNumToMove  =    picNumPred   – (abs_diff_pic_num_minus1 +1)

                    Find the frame with frame_num = ‘picNumToMove’ in reference list

/* place the frame picNumToMove at the slot with RefIdxLX, free the slot by  moving                            right-aligned reference pictures to right

                          num_ref_idx_LX_active_minus1+1 is the size of reference list 

                     */

                     RefPicListX  = 0 

for( idx=num_ref_idx_LX_active_minus1+1; idx>RefIdxLX;idx–)

RefPicListX[idx] = RefPicListX[idx-1]

 

RefPicListX[RefIdxLX] = reference frame with frame_num eq.  picNumToMove

  Read modification_of_pic_nums_idc   // read next reordering command

RefIdxLX = picNumToMove   // set the new pivot

picNumPred = picNumToMove  // frame_num of moved picture

           }

 

 

Example  (progressive case):

Let’s consider the following sequence in display and encoding orders (numbers denote POCs, POC incremented by 2 each frame, B frames not used for reference):

Display order:         I00   b21    b41     P61    b82    b102   P122

Encoding order:      I00    P61    b21   b41    P122   b82     b102

Note: superscripts are frame_num, frame _num is not incrmented if the previous frame is not used for reference

At P61 frame the reference list is L0={I00}

At b21 and b41 frames  the reference lists: L0={I00},  L1={ P61}

At P122 frame the reference list   L0={ P61,I00}

 

Let’s suppose that for the frame P122 the reference I00 is more beneficial, hence it’s worth to swap I00 and P61 in the list L0 (because ref_idx=0 takes 1 bin and ref_idx=1 takes 3 bins). Therefore in each slice header of P122 the following parameters are signaled:

ref_pic_list_modification_flag_lx = 1       // reordering command is present

modification_of_pic_nums_idc = 0       // to address the frame P61 which should be moved and I00 is inserted.

        modification_of_pic_nums_idc = 3    // signal to terminate  

Thus, the modified  reference list for  P122 is L0={ I00, P61}, so the distant reference I00 becomes the immediate.

Leave a Reply

Your email address will not be published. Required fields are marked *