How Count AVC/H.264 Video Frames in MPEG-2 Systems (Transport Stream) file?

Transport stream is commonly comprised by several elementary (“atomic”) streams (e.g. video and audio) and  each elementary stream is specified by unique number PID which is present at the header

of each transport packet. To find the PID of video stream i suggest exploiting tsinfo utility from open-source tstools library.  The official site of tstools

                                                  https://code.google.com/p/tstools/

Example of using tsinfo:

tsinfo    test.ts

Reading from test.ts

Scanning 1000 TS packets

Packet 1 is PAT

Program list:

    Program 1 -> PID 03e8 (1000)

Packet 2 is PMT with PID 03e8 (1000)

  Program 1, version 0, PCR PID 0810 (2064)

  Program streams:

    PID 0100 ( 256) -> Stream type 1b ( 27) H.264/14496-10 video (MPEG-4/AVC)

    PID 0101 ( 257) -> Stream type 0f ( 15) 13818-7 Audio with ADTS transport syntax

        ES info (6 bytes): 0a 04 65 6e 67 00

        Languages: eng

Found 20 PAT packets and 10 PMT packets in 1000 TS packets

 

So, according to the above example’s report the video stream is indeed present in test.ts file and it is in H.264/AVC format with PID = 256 (or 100 in hex).

MPEG-2 Systems standards says: Each AVC access unit shall contain an access unit delimiter NAL Unit

In other words each AVC/H.264 video frame commences with Access Unit Delimiter (NAL type = 9). Hence the number of video frames is equal to the number of Access Unit Delimiters (AUD).

Thus, our task is to count AUDs. Each AUD is NAL unit and consequently it is prefixed by the start code 0x000001 or 0x00000001.

However, PES header is also prefixed by same start code 0x000001. How can one distinguish between the AUD start code and the PES one? It depends on the first bit followed by the start code.

AVC/H.264 mandates that the first bit of each NAL header (i.e. the first bit after the start code) is “0” (it’s called as forbidden_zero_bit) and the purpose of this zero bit is actually to avoid any  conflict

with MPEG-2 system start codes (i.e. PES header start codes) since the first syntax element after the PES start code is stream_id and all stream_id values are of the form 1xxx yyyy   (i.e.  beginning with “1”).

Notice that MPEG-2 adds more restriction which simplifies the detection of video frame starts. MPEG-2 requires that each AVC/H.264 video frame starts with PES header. Therefore, the first start code

belongs to the PES header and the second one belongs to AUD respectively (for simplicity i ignore the case when the start code is emulated within the private data of PES header or within  ts-packet adaptation field).  

In the ts-packet header there is a special flag payload_unit_start_indicator, if this bit is “1” then PES header must present.  Thus AUD can be present but not with necessity since a video frame can be split into multiple PES packets and in such case AUD is present only at the first PES packet.

Let’s outline video frame counting algorithm:

For each ts-packet Do:

                If the packet is video (i.e. its PID equals to the video PID reported by tsinfo) and payload_unit_start_indicator =”1”  (i.e. PES header is present)  Then

                       Look for the first start code 0x000001  – this should be the PES start code  (the case of too long adaptation_field is ignored for simplicity)

                       Look for the second start code 0x000001 (for simplicity i ignore the case when PES header is too long and video start code is actually located in the next video ts-packet)

                       Read the first byte after the second start code and extract nal-type (extract bits 0..5 by “anding” with 0x1f)

                      If nal-type is 9 Then

                                 AUD is detected and increment video frame counter

                     EndIf

 

Notice that in non-video ts-packets (e.g. audio ts-packets) start codes can be emulated therefore we filter all non-video packets in the above algorithm.