Non Reference Video Quality Assessment

Appendix:  blurriness as non-reference Video Quality Metric

 

Three necessary conditions for each Video Quality Metric (VQM) to meet:

1) symmetry: VQM(x, y) = VQM(y, x);

2) boundedness:  0 ≤ VQM(x, y) ≤ 1 (or 100)

3) unique maximum: VQM(x, y) = 1 if and only if x = y.

 

Non-Reference Image Quality Assessment models attempt to estimate the visual quality of a distorted image without a corresponding reference image.

The advantages of Non-Reference Quality Assessment is obvious especially in media-streaming applications, where only the compressed (distorted) video is available, and thus quality judgments must be made without access to the reference.

One of applications of Non-Reference Video Quality Assessment is in Client-Server communication if input video gets severely distorted then Client notifies Server and Server would provide a lower bitrate (sometimes with a lower resolution).

The vast majority of Non-Reference Video Quality methods attempt to detect and quantify specific types of distortion, induced by compression, such as blurring, blocking, ringing, or other forms of noise.  These algorithms are often called distortion-aware.

Notice that recent standards like AV1 or VVC contain in-built de-blocking and de-ringing filters which eliminate blocking and ringing impairments (ringing usually appears near sharp edges), therefore distortion-aware techniques are more suitable for JPEG, JPEG2000 or for obsolete standards like MPEG where in-built filters are absent. For example, ringing occurs in wavelet based compression systems such as JPEG-2000.

However, blur is universal compression-induced distortion which might be present in all lossy compressed video.

 

Outline of distortion-aware method:

1) find some local feature X(i,j), which exhibits different visual distortions inflicted by compression (e.g. blurring), where i, j denotes the position of the pixel

2) use X(i,j) to assess local distortion Ld(i, j)

3) average Ld(i, j)  (or use other pooling technique) over the whole image to get a overall distortion metric Dm 

4)  use Dm to predict image quality score Ps which is consistent with human perception

 

Notes:

• Video Quality Metric estimates the similarity between two signals. However, if one of the signals is the pristine and another the distorted then VQM represents the quality of the distorted signal. If both signals are distorted then VQM is a measure of their similarity.

 

• Frankly speaking VQM is not a metric in mathematical sense since the triangle inequality is not necessarily met.

 

• If x and y are poor quality images and VQM(x,y)=1 then both the images are called visually lossless (not necessarily mathematical identical). What’s visually losslessness – nobody can confidently say what image is better. The sign ‘=’ denotes visually lossless.

 

• VQM(x,y) =1 means that most of viewers would say that two images ‘x’ and ‘y’ are similar in quality (visually identical).
  For example, let ‘x’ is the pristine image and ‘y’ is the distorted, then VQM(x,y)=50% means that most of viewers would say the image ‘y’ is significantly worse in quality relative to the pristine image ‘x’.

 

Appendix: blurriness as non-reference Video Quality Metric

In the paper “A no-reference perceptual blur metric”, Marziliano, Pina, et al., 2002 ,  the non-reference video quality metric, based on blurriness, is described.

This metric was realized in ffmpeg, in filter-complex sub-tool (named ‘blurdetect‘).

 

Example [ compute blurriness for the first 200 frames and store the blurriness of each frame in a separate file ‘slava-blur.txt‘]:

ffmpeg.exe -i crowd.h264 -filter_complex “blurdetect=block_width=32:block_height=32:block_pct=80,metadata=print:file=slava-blur.txt” -frames 200 -f null – 

the file ‘slava-blur.txt’ contains blurriness value per frame (lavfi.blur):

frame:0 pts:0 pts_time:0
lavfi.blur=4.103671
frame:1 pts:48000 pts_time:0.04
lavfi.blur=4.122499
frame:2 pts:96000 pts_time:0.08
lavfi.blur=4.122894
frame:3 pts:144000 pts_time:0.12
lavfi.blur=4.068152
frame:4 pts:192000 pts_time:0.16
lavfi.blur=4.120326
frame:5 pts:240000 pts_time:0.2
lavfi.blur=4.112469
frame:6 pts:288000 pts_time:0.24
lavfi.blur=4.074845

          ….

frame:199 pts:9552000 pts_time:7.96
lavfi.blur=4.222641
frame:200 pts:9600000 pts_time:8
lavfi.blur=4.218632

 

Note: if we remove ‘metadata=print:file=slava-blur.txt‘  then the average blurriness is printed only

ffmpeg.exe -i crowd.h264 -filter_complex “blurdetect=block_width=32:block_height=32:block_pct=80” -frames 200 -f null – 

[Parsed_blurdetect_0 @ 000002c731a32900] blur mean: 4.2147894

 

I tested that the ffmpeg blurriness is monotonically decreasing function of bitrate, but not linear.

The blurriness of the original yuv-sequence:  blur mean: 4.1349611

 

10Mpbs:

 ffmpeg.exe -i fifa_hp_cbr16_10M.h264 -filter_complex “blurdetect=block_width=32:block_height=32:block_pct=80” -frames 200 -f null –

  …

blur mean: 4.5758980

 

 

15 Mbps:

ffmpeg.exe -i fifa_hp_cbr16_15M.h264 -filter_complex “blurdetect=block_width=32:block_height=32:block_pct=80” -frames 200 -f null –

…

blur mean: 4.4421685

 

20 Mbps:

   ffmpeg.exe -i battlefield_hp_cbr16_20M.h264 -filter_complex “blurdetect=block_width=32:block_height=32:block_pct=80” -frames 200 -f null –

 blur mean: 4.3732008   {close to blurriness of the original sequence}