The Introduction of AVS Part-1: System

The introduction of AVS Part-1: System

AVS Part-1 converts single/multi-channel audio and video stream, or other basic data stream into single/multi-channel multiplexed stream for storage and transmission. System encoding of AVS is composed of two components: the Transport Stream and the Program Stream which are optimized for different applications respectively. AVS defines encoding syntax which is necessary for Synchronous decoding and displaying audio, video information. The syntax ensures that overflow and underflow not appears in the data buffer of decoder. The syntax contains at least one of following two types of information: a) The information describes the relative time stamp of the encoded audio/video data and the bit rate of the stream; b) The information describes the time stamp of the encoded audio/video data and the time stamp of the transmission of the stream. Meanwhile, the Transport Stream and the Program Stream are the multiplexing streams with the package. The multiplexing process of the elementary stream of single audio and video is shown as Figure 1. Video and audio can be encoded according to AVS Part2, Part 7, Part3 and part10 encoding standards. Then the compressed data are packed and becomes a PES (packetized elementary stream).

Video Data Video Video PES Packetizer Encoder PS Program Stream MUX Audio Data Audio Audio PES Packetizer Encoder

TS Transport Stream MUX

Extent of Systems Specification

Figure 1. Multiplexing process diagram PES is an abbreviation for packetized elementary stream. Its syntax and semantics shall be used to encapsulate the audio and video elementary stream information. The PES syntax is used to convey the Presentation Time-Stamp (PTS) and Decoding Time-Stamp (DTS) information required for decoding audio and video information with synchronism. Figure.2 shows the syntax structure of PES. Packet PES Optional stream start code packet PES PES data byte Id prefix length header 24 8 16

PES data PES stuffing PES original or 7 marker Optional ‘10’ scrambling alignment Copyright header byte priority copy bits field control indicator data length (0xFF )

2 2 1 1 1 1 8 8 m*8

DSM trick Additional PES PTS DTS ESCR ES rate PES CRC mode copy info extension

33 42 22 8 7 16

5 marker Optional bits field

PES program PES length PES packet packets P-STD private of PES extension header sequence buffer data extension field field data field control 128 8 8 16 7

Figure.2 syntax of packetized elementary stream

1. Program Stream

Program Stream is a single stream composed of one or more PES packets with the common timestamp, and designed for relatively error-free environment, and adapted to the interactive multimedia applications and so on. The packet length of the Program stream is variable. The rate of Program Stream can be constant or variable. In any case, the rate of basic stream which Program Streams contains is constant or variable. The rate of Program stream is defined by the value and the location in the system clock reference rate (SCR) field and the mux-rate field. Figure 3 displays a prototype of audio / video program stream decoding system of AVS standard, but its structure is not unique. The Program Stream decoder prototype shown in Figure 3 is composed of system, video and audio, which accord with AVS Part1, Part2 and Part3. The prototype decoder of Program Stream accepts the Program Stream as input, extracts information from the stream depending on the Program Stream decoder. Then, the Program Stream decoder demultiplexes the multiplexed stream, outputs elementary stream which becomes the input of the audio decoder and the video decoder. The outputs of the audio/video decoder are decoded audio and decoded video signals. Program Stream can be divided into two layers: System Layer and Compression Layer. The input stream of the Program stream decoder consists of a System Layer including Compressed Layer. As for the audio decoder and the video decoder, the input stream only contains Compressed Layer. Decoded video Video decoder

Channel Channel Program Stream Clock control decoder decoder

Decoded audio Audio decoder Program Stream

Figure.3 Decoder of Program Stream Program Stream encoder can combine one program consisting of one or more basic streams into a single stream. In this case, the data of the basic streams and the information which enable the synchrony play of those streams are multiplexed together. A Program Stream contains one or more basic streams which are multiplexed together and belong to a same program. PES packages carry the data of Basic stream. The arrangement of the PES packets group is the packet header at first and the packages following. The PES packet header begins with a 32-bit code, which also identifies which stream the packet data belongs to. In the Program Stream, PES packets are combined into a group. PES Packet group starts with packet header, followed by none or some PES Packets. Packet header storing time and bit rate information begins with a start_code of 32-bit. The structure of Program Stream is shown as Figure 4.

AVS XXX program Packet Packet Packet 1 Packet 2 Packet 3 Packet n stream header … header

Program Packet Packet packet start System PES PES PES ‘01’ SCR multiplexing stuffing stuffing Packet layer code header packet 1 packet 2 … packet n rate length byte

32 2 42 22 5 3

upper upper upper System Header Fixed CSPS Video Audio limit of limit of limit of N loop start code length marker flag lock flag lock flag rate audio Video

P-STD P-STD Stream proportion of the ‘11’ Buffer size ID buffer … limits boundaries 8 2 1 13

Figure 4. The structure of Program Stream Program Stream Map (PSM) describes the basic streams and the interrelationships between them, as Figure.5 displayed.

Packet program current program program map N loop start code stream map next stream map stream stream Id descriptor prefix length indicator version info length

24 8 16 1 2 5 7 16

elementary stream map N loop CRC 32 length 16 32

stream elementary elmentary stream N loop type stream ID info length descriptor

8 8 16

Figure 5. Program Stream Map

2. Transport Stream

Transport Stream is composed of one or more single programs, forming a single stream. These programs may contain one or more different clock reference, and PES packets consisting of several Elementary Stream in the same program share a common clock reference. Transport Stream is designed for resisting possible prejudicial errors or noises in the media storage or transmission environment. And the length of the Packet of Transport Stream is 188 bytes. The rate of Transport Stream can be constant or variable. In any case, the rate of basic stream which Program Streams contains is also constant or variable. The rate of Program stream is defined by (transport_rate) field, which are embedded in each streams. Transport Stream is designed with the principle that certain operations can be performed with minimum spending. These operations include: a) Obtaining encoded data from a program in the Transport Stream, decoding and displaying it, as shown in Figure 6. b) Extracting Transport Stream packets from one program, generating a new stream only containing the program as the output, as shown in Figure 7. c) Extracting Transport Stream packets belonging to one or more programs from one or more Transport Streams, and producing a new transport stream with these extracted packets. d) Extracting the content of a program from the Transport Stream and generating a Program Stream with the extracted program. e) Converting a Program Stream into a Transport Stream, transmitting it in a lossy and noisy environment. And then rebuild an effective program stream, in some cases exactly the original program stream. . Figure 6 and Figure 7 describes a prototype of demultiplexing and decoding system with Transport Stream as input. Figure 6 illustrates the operation a), in which a Transport Stream is demultiplexed and decoded directly. The structure of Transport Stream can be divided into two layers: System Layer and Compression Layer. The input stream of Transport Stream decoder is composed of a System Layer contains Compressed Layer. As for the audio decoder and the video decoder, the input stream only contains Compressed Layer. Figure. 6 illustrates a Transport Stream decoder prototype including video and audio decoders afforded by the AVS standard. It’s not mean that the prototype issued by AVS standard asks the strictly limit for the design of Transport Stream decoder, and the decoder is not unique. In fact, the non- audio/video data can also be used. Figure 7 shows another situation b) in which the Transport Stream contains one or multiple programs is transformed into a Transport Stream only including a single program. In this case, the re- multiplexing operation may be needed to correct transport_rate value for compensating the change of transmission rate in bit stream.

Decoded video Video decoder

Channel Transport Stream Channel demultiplexer & Clock control decoder decoder

Decoded audio Audio decoder Transport stream including one or more programs

Figure 6. Transport Stream demultiplexing and decoding prototype

Channel Channel Transport Stream decoder demultiplexer & decoder

Transport stream including Transport stream including multi-programs single program

Figure 7. Transmission multiplexing prototype

AVS Transport Stream coding allows one or more programs are packaged in a stream. Each basic stream and the information used to synchronize the data and display basic streams of each program are multiplexed together. The Transport Stream is shown in Figure 8. A Transport Stream consists of one or more programs. The elementary stream of video and audio is composed of one or more access units. Basic stream data are carried by the PES packet. A PES packet is composed of header and data. PES packets are inserted into the Transport Stream packets, the first byte of the header of each PES packet is placed at the location of first available payload in the Transport Stream packets. PES packet header begins with a 32-bit start code, the start code also identifies which stream or stream type the package belongs to. Transport stream packets can be empty, which are used to fill the Transport Stream, may be inserted or deleted during re-multiplexing process.

188 byte s

head payload head payload head payload

Transport Payload Transport Adaptation Sync Transpor PEIT Continuity Adaptatio error unit start scramblin field byte t priority indicator counter n field indicator indicator g control control 8bits 1 1 1 13 2 2 4

Elementary adaptation Random Discontinuity stream PCR,OPCR Optional Stuffing field access indicator priority etc. flag field byte length indicator indicator 8 1 1 1 5

Private Adaptation Splice Private Specific field Optional PCR OPCR count transport 3 flags transport extension field down data data length length 42 42 8 8 8 3

Piecewise Splice ltw_valid Ltw offset DTS_next_au rate type

1 15 2 22 4 32

Figure 8. The structure of Transport Stream

Transport Stream Map (PSM) describes the basic streams and their interrelationships, as shown in Figure 9. section current last section program version section CR table ID syntax ‘0’ next section length number number number PEID indicator indicator number 8 1 1 2 12 16 1 5 8 8 8 3 13 4

program N loop N loop CRC 32 info length descriptor

stream elementary ES info N loop type PEID length descriptor

8 3 13 4 12

Figure 9. Transport Stream Map The Program Stream and the Transport Stream are designed for different applications. The definition of Program stream and the Transport Stream do not strictly abide by the Layering models; they can be interchangeable with each other, but not a subset or superset of each other. A program can be extracted from a Transport Stream and an effective program stream can be created by it. The conversion between these two kinds of streams can be completed according to the public interchange format of PES Packet, but not all the fields the program stream needs are included in the transport stream; some fields need to be exported.

Reference

GB/T 20090.1 Information technology - Advanced coding of audio and video – Part 1: System, Chinese AVS standard.