VVenC Open Source Implementation of VVC Codec

VVEnC

How get VVEnC? 

How Build VVEnC in Windows?

How Run VVEnC? 

VVDec

How get VVDec? 

How Build VVDec in Windows? 

How Run VVDec? 

Appendix: Subjective Comparison between HEVC and VVC

Appendix: Objective Comparison between HEVC and VVC

Appendix: Objective Comparison between VVenC and other codecs

Appendix: Use Case – Percentage of Partitions

Appendix:  Multiple Transform Selection

Appendix:  On Myth VVC is by 50% better HEVC

 

 

VVEnC 

VVenC (the Fraunhofer Versatile Video Encoder)  is a freely available optimized software encoder implementation of the VVC standard.  The source code is available on GitHub (https://github.com/fraunhoferhhi/vvenc) under a permissive 3-clause BSD copyright license for both commercial and noncommercial.  

Notes:

• According to the paper “OpenVVC: a Lightweight Software Decoder for the Versatile Video Coding Standard”, by Thomas Amestoy et al., 2022 , the computational complexity of the VVC reference encoder has increased by a factor 8 and 27 compared to the HEVC reference encoder in inter and intra coding configurations, respectively.

 

• In VVC four in-loop filters are performed in sequel on the reconstructed samples in order to reduce the visual artifacts:inverse mapping (LMCS) – redistribute the reconstructed luma samples to a smaller range of values.Deblocking Filter (DBF)  – applied on block boundaries, reducing the blocking artifactsSample Adaptive Offset (SAO) – to filter the ringing artifacts and enhance the perceptual video quality.Adaptive Loop Filter (ALF) – Wiener filtering to minimize the Mean Squared Error (MSE) between original and reconstructed samples.

 

How get VVEnC? 

git clone https://github.com/fraunhoferhhi/vvenc  

and vvenc folder is created with sources

 

How Build VVEnC in Windows? 

in vvenc folder create ‘build’ directory and run ‘cmake’ to build corresponding Visual Studio Solution:

mkdir build

cd build 

cmake .. -G “Visual Studio 16 2019”

 

Visual Studio solution vvenc.sln is created ,  open vvenc.sln with Visual Studio (you can make retarget if you have a more advanced Visual Studio version).

For example choose  Release/x64 and press rebuild all (although we need build actually vvencapp project).

Exe files (e.g. vvencapp.exe) are created in the folder:  vvenc\bin\release-static\

 

How Run VVEnC? 

In the folder  vvenc\bin\release-static\  run   ‘vvencapp.exe -h’ and you get the full help info, i present selected options:

-i,         original YUV input file name or ‘-‘ for reading from stdin

-s,        specify input resolution (WidthxHeight)

-c,        set input format (yuv420, yuv420_10, yuv420_10_packed)

-r,        temporal rate (framerate numerator) e.g. 25,30, 30000, 50,60, 60000

-f,        max. frames to encode [ default  –  all]

-o,       bitstream output file name

--preset       select preset for specific encoding setting (faster, fast, medium, slow, slower), default medium

-b,               bitrate for rate control (0: constant-QP encoding without rate control) in units of bits/second

-q,               quantization parameter, QP (0-63) if the bitrate is set to ‘0’

--qpa          enable perceptually motivated QP adaptation, XPSNR based (0:off, 1:on)

-t,               number of threads, defaut: 4 if picture size < 720p: 4,otherwise 8

-g,             GOP size of temporal structure (16,32)

-rt,             intra refresh type (idr,cra)

-rs,            intra period/refresh in seconds

--tiles        set number of tile columns and rows, default 1×1, i.e. single tile

-p,            number of rate control passes (1,2)

 

Example [ bitrate 5Mbps, frame rate 50fps, preset fast, single pass rate control (-p 1), 2×2 tiling corresponding to 4 threads (-t 4), intra period 1s (-rs 1), key frames are IDRs (-rt idr)]

 

vvencapp.exe -s 1920×1080 -c yuv420 -r 50 -i crowdrun1080p50fps.yuv   --preset fast     -b  5000000  --qpa 0 -t 4 -g 16  -rt idr -rs 1 --tiles 2×2 -p 1 -o  crowdrun1080p50fps5Mbps.266

…

POC    0 TId: 0 (  IDR_N_LP, I-SLICE, QP 32 )    1298760 bits [Y 34.1006 dB    U 36.3206 dB    V 37.0098 dB] [ET     5 ] [L0 ] [L1 ]

POC   16 TId: 0 (     TRAIL, B-SLICE, QP 38 )     490168 bits [Y 30.9347 dB    U 35.1367 dB    V 35.7650 dB] [ET     2 ] [L0 0 ] [L1 0 ]

POC    8 TId: 1 (      STSA, B-SLICE, QP 40 )     245712 bits [Y 29.4474 dB    U 34.7956 dB    V 35.2190 dB] [ET     1 ] [L0 0 16 ] [L1 16 0 ]

POC    4 TId: 2 (      STSA, B-SLICE, QP 44 )      94448 bits [Y 28.2286 dB    U 34.6254 dB    V 34.9169 dB] [ET     1 ] [L0 0 8 ] [L1 8 16 ]

POC    2 TId: 3 (      STSA, B-SLICE, QP 45 )      38512 bits [Y 28.6776 dB    U 35.1850 dB    V 35.6019 dB] [ET     1 ] [L0 0 4 ] [L1 4 8 ]

POC    1 TId: 4 (      STSA, B-SLICE, QP 46 )       8560 bits [Y 29.7323 dB    U 35.6221 dB    V 36.2087 dB] [ET     0 ] [L0 0 2 ] [L1 2 4 ]

POC    3 TId: 4 (      STSA, B-SLICE, QP 46 )       8664 bits [Y 28.0818 dB    U 34.9032 dB    V 35.2751 dB] [ET     0 ] [L0 2 0 ] [L1 4 8 ]

POC    6 TId: 3 (      STSA, B-SLICE, QP 45 )      38416 bits [Y 27.8574 dB    U 34.5796 dB    V 34.8478 dB] [ET     1 ] [L0 4 0 ] [L1 8 16 ]

POC    5 TId: 4 (      STSA, B-SLICE, QP 46 )       8016 bits [Y 27.7567 dB    U 34.6245 dB    V 34.8996 dB] [ET     0 ] [L0 4 0 ] [L1 6 8 ]

POC    7 TId: 4 (      STSA, B-SLICE, QP 46 )       8176 bits [Y 28.0678 dB    U 34.6557 dB    V 35.0125 dB] [ET     0 ] [L0 6 4 ] [L1 8 16 ]

….

POC   29 TId: 4 (      STSA, B-SLICE, QP 45 )       9744 bits [Y 28.1482 dB    U 34.4780 dB    V 34.9059 dB] [ET     1 ] [L0 28 24 ] [L1 30 32 ]

POC   31 TId: 4 (      STSA, B-SLICE, QP 45 )       9648 bits [Y 28.8903 dB    U 34.7992 dB    V 35.2850 dB] [ET     1 ] [L0 30 28 ] [L1 32 30 ]

POC   48 TId: 0 (IDR_W_RADL, I-SLICE, QP 34 )    1018552 bits [Y 33.0380 dB    U 35.5083 dB    V 36.1820 dB] [ET     5 ] [L0 ] [L1 ]

POC   40 TId: 1 (      RADL, B-SLICE, QP 40 )     286976 bits [Y 29.0697 dB    U 34.4217 dB    V 34.8734 dB] [ET     1 ] [L0 48 ] [L1 48 ]

POC   36 TId: 2 (      RADL, B-SLICE, QP 46 )      80680 bits [Y 26.6824 dB    U 33.5366 dB    V 33.7131 dB] [ET     1 ] [L0 40 48 ] [L1 40 48 ]

…..

POC   77 TId: 4 (      STSA, B-SLICE, QP 46 )       7736 bits [Y 27.6559 dB    U 34.1883 dB    V 34.5285 dB] [ET     1 ] [L0 76 72 ] [L1 78 80 ]

POC   79 TId: 4 (      STSA, B-SLICE, QP 46 )       7520 bits [Y 28.9766 dB    U 34.6757 dB    V 35.1323 dB] [ET     1 ] [L0 78 76 ] [L1 80 78 ]

POC   96 TId: 0 (IDR_W_RADL, I-SLICE, QP 34 )    1023640 bits [Y 32.8973 dB    U 35.5821 dB    V 36.2832 dB] [ET     7 ] [L0 ] [L1 ]

POC   88 TId: 1 (      RADL, B-SLICE, QP 39 )     313936 bits [Y 29.8437 dB    U 34.4309 dB    V 35.0289 dB] [ET     2 ] [L0 96 ] [L1 96 ]

POC   84 TId: 2 (      RADL, B-SLICE, QP 44 )     111792 bits [Y 27.7482 dB    U 33.7537 dB    V 34.2169 dB] [ET     2 ] [L0 88 96 ] [L1 88 96 ]

…

Total Time: 682.773 sec. Fps(avg): 0.732 encoded Frames 500

 

Notes:

IDRs might inflict flickering in visual quality, since Y-PSNR of regular frames is 26-29dB while Y-PSNR scores are around 32-33dB.

 

VVDec 

VVdeC (the Fraunhofer Versatile Video Decoder) is a freely available optimized software decoder implementation of the VVC standard. The source code is available on GitHub (https://github.com/fraunhoferhhi/vvdec) under a license BSD, similar to that of VvenC.

The VVdeC package contains a simple and easy-to-use C library interface and a standalone decoder application capable of decoding elementary VVC bitstreams into raw YUV video data.

 

How get VVDec? 

git clone https://github.com/fraunhoferhhi/vvdec

and vvdec folder is created with sources

 

How Build VVDec in Windows? 

in vvenc folder create ‘build’ directory and run ‘cmake’ to build corresponding Visual Studio Solution:

 

mkdir build

cd build 

cmake .. -G “Visual Studio 16 2019”

 

Visual Studio solution vvdec.sln is created,  open vvdec.sln with Visual Studio (you can make retarget if you have a more advanced Visual Studio version).

For example choose  Release/x64 and rebuild the vvdecapp project.

Exe files vvdecapp.exe is created in the folder:  vvdec\bin\release-static\

 

How Run VVDec? 

-b       bitstream input file, 266

-f           max. frames to decode (default: -1 all frames)

-o         yuv output file

--y4m   force y4m output

 

Example: 

vvdecapp.exe -b crowdrun1080p50fps5Mbps.266 --y4m -o crowdrun.y4m

 

Note: there is an alternative  OpenVVC decoder (sources, highly optimized)

There is the report about performance of optimized VVC decoder on Intel®Xeon®W:

AHG16: Performance of a reasonably fast VVC software decoder

Performance of optimized VVC decoding is above 60fps for all testing sequences. According to profiling of optimized VVC decoder, the lion’s share of decoding time is Inter Prediction and ALF. So, disabling ALF seems reasonable.

 

 

 

Appendix: Subjective Comparison between HEVC and VVC

• The interesting paper “Suitability of VVC and HEVC for Video Telehealth Systems”, Muhammad Arslan Usman et al., 2020

In the paper DMOS (where DMOS = MOS_pristine – MOS_compressed) scores are reported for regular video sequences, like Lion, Cheetah, Rainbow etc. (not related to telemedicine), for both HEVC and VVC.

The bottom line of the subjective testing in the paper between HEVC and VVC:

“Our results show that VVC can reduce bitrates up to 40%, while maintaining same or even better visual quality as HEVC”

Elaboration: a relationship between DMOS scores and QP values are revealed in the figures 6 (HEVC) and 7 (VVC).

Especially the figure 8 of the paper (attached below) is interesting, where the comparison of DMOS scores between HEVC and VVC is presented:

• The results of DMOS scores on 8K (BT-2020) video, taken from “Perceptual Quality Assessment of HEVC and VVC Standards for 8K Video“, by  Charles Bonnineau et al., 2021

 

 

Appendix: Objective Comparison between HEVC and VVC

• In the paper “AN OBJECTIVE EVALUATION OF CODECS AND POSTPROCESSING TOOLS FOR 8K VIDEO COMPRESSION”, by Charles Bonnineau et al., 2020
  Results of the comparison of HEVC and VVC on 8K content (with different spatial and temporal activity) are revealed.

8K Test sequences:

Results:

Roughly speaking, VVC provides same video quality as HEVC with the bitrate smaller at least by 25%.

 

• There is more recent paper of the same authors, where video quality results of HEVC vs. VVC are revealed for 8K (10bpp or BT2020) sequences:  “Perceptual Quality Assessment of HEVC and VVC Standards for 8K Video“, by  Charles Bonnineau et al., 2021

 

In HEVC to get “visually lossless” (visual distortions are non-perceivable, usually VMAF scores above 94) for 8K video (BT-2020 or HDR) one needs the bitrate around 150 Mbps, while for VVC one needs about 60 Mbps to cross the visually-lossless barrier.

 

Note:  With adoption binary and ternary CU partitions in VVC the number of choices exploded.  Therefore fast CU partition algorithms are necessary for coding VVC.  Roughly speaking there are two categories of fast heuristic CU division algorithms: based on image features (like smoothness) and based on machine learning.

 

 

Appendix: Objective Comparison between VVenC and other codecs

From the paper “A VVC/H.266 Real-time Software Encoder for UHD Live Video Applications“, by Sergio Sanz-Rodriguez et al., 2022

VVenC SW encoder has best R-D curves among other encoders in both PSNR and VMAF metrics:

 

 

Appendix: Use Case – Percentage of Partitions

VVC has 6 partition modes:

 

QuadTree (QT) partition
Vertical Binary Tree (VB)
Horizontal Binary Tree (HB)
Vertical Ternary Tree (VT)
Horizontal Ternary Tree (HT)
Non-Partitioning (NP)

According to the paper “Intra Complexity Control Algorithm for VVC” by Zhengjie Shu et al., ternary partitions are non-popular for all QPs:

 

 

 

 

 

 

Appendix:  Multiple Transform Selection

VVC supports so called “Multiple Transform Selection” (MTS). Three transform types are used in VVC: DCT–II, DCT–VIII and DST–VII.
They are integer approximation of the following original transforms:

the above table taken from “Performance Analysis of Optimized Versatile Video Coding Software Decoders on Embedded Platforms”, by Anup Saha et al., 2021

 

 

 

Appendix:  On Myth VVC is by 50% better HEVC

The claims that VVC is by 50% better than HEVC have not confirmed.
According to the paper “Comparative Rate-Distortion-Complexity Analysis of VVC and HEVC Video Codecs“, by ALEXANDRE MERCAT et al.:

VTM (VVC reference encoder) improves the average coding efficiency over HM (HEVC reference encoder) depending on quality metric, by 23.0-23.9% under the AI (All Intra) condition, 33.1-36.6% under the RA (Random Access, five-layered hierarchical GOP structure) condition,   and 26.7-29.5% under the LB (Low Delay) condition.

However, encoding times of VTM:

for AI are 34.0 times higher than HM
for RA 8.8 times higher than HM
for LB 7.5 times higher than HM

 

VTM decoder times are 1.8x greater than HEVC, regardless to AI, RA or LB.

  RitualDance 1080p

 

 

Tango (4096×2160)

C

 

Complexity analysis (Table 8 from the paper):

Legend:

ME/MC – motion estimation and compensation

IP – intra prediction

LF – loop filers

EC – entropy coding (CABAC)

TR/Q – transform quantization, selection a good partition

          kcpp   – K-cycles per pixel.

Most of processing (50%-60%) is consumed by Motion Estimation (ME)

 

 

VVC Decoder complexity

ED – entropy decode

IQ/IT inverse quantization and transform

MC – motion compensation

LF – loop filters