Introduction
The purpose of the list reordering feature of H264/AVC is to show how reference frame reordering impacts on coding efficiency.
This document describes in details how to make the reference list reordering in order to achieve better coding efficiency. The H.264/AVC supports special reference list reordering commands (signaled in the slice header) and which are applied on both reference lists: L0 (previous frames) and L1 (future B-frames) . The same reordering commands are performed in Encoder and Decoder, otherwise a drift (or async) between Encoder and Decoder is inevitable.
Note: This shuffling by reordering commands is temporary and is valid only within a given frame (more precisely within a given slice).
The reference picture indexes ref_idx_l0 and/or ref_idx_l1 (if B frames are used) is signaled for each MB inter partition, excepting the skip or the direct modes (in both skip and direct modes ref_idx is derived from previous values of ref_idx).
The parameter ref_idx_l0[l1] is binarized as a truncated Exp-Golomb code and then forwarded to Arithmetic Engine. Large values ref_idx_l0 or ref_idx_l1 require much more bins than low values. Indeed, if the reference lists consists of four references: Ref0 (1 bin), Ref1 (3 bins), Ref2 (3 bins) and Ref3(5bins).
When most the distant reference is more beneficial than the immediate previous one. The situation of flash lights is not uncommon, especially in gaming content. In cases of flash lights the reference Ref3 is a good candidate for reference, despite its distance to the current frame, since the previous frames are “corrupted” by flash lights:
The problem is that Ref3 requires 5 bins while Refo 1 bin. Hence Ref3 index is bits-consuming. However, if we reorder the reference list for all slices within the current frame such and to place Ref3 at the first position (with index 0 and 1 bin) and Ref0 is placed in the last position respectively (the index 3 with 5 bins). In such case a gain in coding efficiency is expected. So, it’s worth to reorder the reference list as follows:
Ref3, Ref1, Ref2, Ref0
Notice that reordering commands are signaled in the slice header, hence regions “corrupted” by flash lights are reordered.
Reference List Reordering Syntax
Reordering is signaled in the slice header with the following syntax elements (here the suffix ‘x’ denotes the refrence list L0 or L1) :
ref_pic_list_modification_flag_lx – if 1 then reordering commands are present in the slice header and are conducted by both Encoder and Decoder in the same way (to avoid a drift between Encoder and Decoder).
abs_diff_pic_num_minus1
modification_of_pic_nums_idc – together with abs_diff_pic_num_minus1 indicates which of the pictures in the reference list is moved. The value 3 means termination process.
Algorithm of Shuffling Reference Pictures
A tacit assumption, a frame_nums of all pictures in reference list are less than 2log2_max_frame_num
// set initial prediction values if field_pic_flag==0 // i.e. the current picture is frame and not a field picNumPred = frame_num of current frame Else // fields, interlace video picNumPred = [frame_num of current frame]*2+1
RefIdxLX=0 Read modification_of_pic_nums_idc // read first reordering command while (modification_of_pic_nums_idc!=3) // ‘3’ stands for termination { Read abs_diff_pic_num_minus1 // compute frame_num of frame to be moved picNumToMove = picNumPred – (abs_diff_pic_num_minus1 +1) Find the frame with frame_num = ‘picNumToMove’ in reference list /* place the frame picNumToMove at the slot with RefIdxLX, free the slot by moving right-aligned reference pictures to right num_ref_idx_LX_active_minus1+1 is the size of reference list */ RefPicListX = 0 for( idx=num_ref_idx_LX_active_minus1+1; idx>RefIdxLX;idx–) RefPicListX[idx] = RefPicListX[idx-1]
RefPicListX[RefIdxLX] = reference frame with frame_num eq. picNumToMove Read modification_of_pic_nums_idc // read next reordering command RefIdxLX = picNumToMove // set the new pivot picNumPred = picNumToMove // frame_num of moved picture }
|
Example (progressive case):
Let’s consider the following sequence in display and encoding orders (numbers denote POCs, POC incremented by 2 each frame, B frames not used for reference):
Display order: I00 b21 b41 P61 b82 b102 P122
Encoding order: I00 P61 b21 b41 P122 b82 b102
Note: superscripts are frame_num, frame _num is not incrmented if the previous frame is not used for reference
At P61 frame the reference list is L0={I00}
At b21 and b41 frames the reference lists: L0={I00}, L1={ P61}
At P122 frame the reference list L0={ P61,I00}
Let’s suppose that for the frame P122 the reference I00 is more beneficial, hence it’s worth to swap I00 and P61 in the list L0 (because ref_idx=0 takes 1 bin and ref_idx=1 takes 3 bins). Therefore in each slice header of P122 the following parameters are signaled:
ref_pic_list_modification_flag_lx = 1 // reordering command is present
modification_of_pic_nums_idc = 0 // to address the frame P61 which should be moved and I00 is inserted.
modification_of_pic_nums_idc = 3 // signal to terminate
Thus, the modified reference list for P122 is L0={ I00, P61}, so the distant reference I00 becomes the immediate.
23+ years’ programming and theoretical experience in the computer science fields such as video compression, media streaming and artificial intelligence (co-author of several papers and patents).
the author is looking for new job, my resume