THE COMPLETE TECHNICAL PAPER PROCEEDINGS FROM: A METHOD OF ANALYZING MPEG DATA IN ENCAPSULATED STREAMS F. Eugene Rohling DVA Group, Inc. Abstract tools for looking at stream usage of Motorola Broadband DigiCipher, Scientific Atlanta This paper describes a method of analyzing Power Key, and other access control encapsulated binary data streams for the streams are usually held close. This makes it purposes of performing detailed message difficult for an MSO to find problems in his analysis. This method evolved from a general local system, especially when he is purpose analysis tool used to analyze radar responsible for operating it. data. It is now being applied to the analysis of MPEG-2 content and access control data Encapsulated MPEG Data delivered both in-band and out-of-band. It is particularly useful for compartmentalizing the The National Access Control Service details of sensitive control and encryption (NAS) owned by Motorola Broadband and information within the MPEG data strea of an operated by AT&T (now Comcast) is an access control system.. excellent example of MPEG encapsulated data. Figure 1 shows the various layers of The method allows users to describe MPEG data. First, the DigiCipher OOB data encapsulated framed data, parsing a binary is encapsulated into MPEG private data data stream, and generating human readable message packets. When it arrives in the output that can be used to analyze and resolve headend, data is then sent from the satellite problems. The template files can be tailored receiving device (IRT) across Ethernet to the and customized to reveal varying levels of out of band modulator (OM). That is, the proprietary and confidential data within the OOB data is carried as an encapsulated MPEG binary stream. data stream within a HITS multiplex through the satellite system. [1] INTRODUCTION Level 2 Digicipher AC Messages This paper identifies a solution that helps Encapsulation MPEG Messages test and field engineers analyze complex Level 1 Digicipher OOB Traffic MPEG data streams. It uses the familiar NAS MPEG Packets access control service as an example of data Encapsulation that has been encapsulated four times when it HITS Transport Stream Visible Layer is received within a headend system. Finally, MPEG Packets it discusses the need for these tools as new technologies emerge. Figure 1 - NAS Encapsulation This paper specifically discusses access control data. Many off-the-shelf tools exist A standard MPEG recording tool such as for analyzing standard MPEG-2 video and the DSTS by Logic Innovations allows you to DOCSIS services. However, access control record the data stream as it is received by the systems are by their nature proprietary, and IRT. But if you want to recover only the data seen by the set-top, then you must remove the such as awk, sed, grep, lex, and perl. But HITS transport stream. converting a 100 MByte file from binary to readable text becomes unwieldy when the Several one-off tools have been built to result can generate many Gigabytes of data detunnel the data, but they are all considered and take significant time to sort through and proprietary by the AC provider. Sometimes filter that data. an MSO has legitimate reasons to determine if his access control system is operating properly It is significantly less time consuming for or if he is receiving all the data his contract analysts to process binary data and extract with NAS provides. only the information they need to do their task. Similar problems exist with Motorola DAC based local access controllers. In this HISTORY case, the problem becomes more urgent because the MSO is responsible for the The problem of analyzing a complex data operation of the DAC. stream that has been multiplexed into many layers is not unique to the cable or MPEG In many systems, access control data is industries. Instrumentation systems during the encapsulated on a TCP/IP network and sent to 1980 to 1995 time frame commonly mixed a modulating device. Rather than data being and multiplexed dissimilar data from many MPEG encapsulated in MPEG, it is now sources within a telemetry or tape recorded MPEG encapsulated within IP. While good data stream. Ethernet tools exist, they to not provide utilities to integrate with MPEG tools. [1] The Link to Radars Compartmenting Data A good example was a radar instrumentation system developed for the F- To give the MSO the tools Motorola 15, F-16, and B-1 aircraft by Lockheed originally used to develop DigiCipher would Georgia under the Advanced Radar Test Bed be giving away the keys to their access control (ARTB) program. The requirements for that kingdom. But to give MSO’s tools that help system required it to visualize and record identify if code objects are spinning, or if TV traffic from up to four MIL-STD-1553 data Guide data is still online, or to identify if bus streams, up to four streams of telemetry channel maps are being provided to their data, several custom low, medium, and high facility are all reasonable requests. speed data streams at an aggregate rate of up to 12 Mbytes/sec. This was a feat for the A legitimate need exists to compartment 1989 designed system. They also required the the visibility of MPEG access control system to be versatile and instrument any of implementation so legitimate users can five radars on the three aircraft. The visualize it operationally without requirements finally required time stamping compromising the access control system. the data to +/- 10 microseconds. Processing Binary Data High Speed Analysis Becomes Key Instrumenting the aircraft, multiplexing Many processing programs exist for data, recording data, and time tagging data processing text. Unix has a wealth of tools was straightforward. Much of it was performed in hardware. But the system Processing Frames of Data proved that reducing and analyzing the data became a significant labor intensive task. The MAcq filter input description was U.S. Air Force engineers likened the task of designed to process nested frames of variable finding a needle in a hay stack. length data in a serial data stream. Figure 3 shows the format of the filter file. Note that This system evolved into the bench top the format of the filter file allows recursion. Radar Instrumentation System (RIM-68) That is, optional filter frames can be nested developed by Flexible Engineering Resources, within a top level scope frame to create the Inc. (FER). This company developed a same data recursion effect often found with method of encapsulating the data in a common software recursion. This is the primary format and a method of parsing the data at benefit of applying MAcq filters to high speeds so a small number of parameters encapsulated data problems. could be visualized in both text and graphic format. The method was coined “MAcq” for Modular Acquisition. FRAME DESCRIPTOR MACQ FILTERING [3] DATA DESCRIPTION The “macq_filter” program performed the Describes Fields within Frame analysis side of this task was called the “MAcq_filter”. It analyzed data for both real- time and post processing. It used “filters” that LENGTH DESCRIPTOR described the encapsulated nature of the data stream to both extract and process the stream Function = How to determine frame length into either human readable form or into derivative streams for off-the-shelf graphic programs to process as shown in Figure 2. Pass Filter Input Output Function = How to Data macq_ Data determine whether this Stream filter Stream frame is of interest. (File) (File) How to output data (Binary, Text, ...) Default = pass binary MAcq Filter File Optional recursive frame (.flt) Figure 2 - MAcq Filter Process Figure 3 - Filter File Format HITS STREAM NAS OOB NAS OOB MPEG PACKETS MPEG PACKETS MPEG MESSAGE PID = NAS OOB CA Stream PID = Other Services MPEG Packets Access Control CA Stream Message to Digicipher PID = Other Services MPEG Packets Devices CA Stream PID = NAS OOB MPEG Packets PID = Other Services PID = NAS OOB CA Stream PID = Other Services Access Control MPEG Packets Message to Digicipher CA Stream PID = Other Services Devices MPEG Packets PID = Other Services PID = NAS OOB PAT PID = Other Services pat_info.flt PID = Other Services FILTER #ifdef SHOW_PAT PID = NAS OOB <When PID = 0> <Output PID information> PID = Other Services #endif END FILTER PID = Other Services CAT PID = Other Services cat_info.flt FILTER #ifdef SHOW_CAT <When PID = 1> <Output PID information> HITS STREAM FILTER FILE NAS OOB FILTER FILE #endif hits_filter.flt nas_mpeg_pkt.flt END FILTER FRAME = MPEG PACKET FRAME = MPEG PACKET <MPEG Packet Description> <MPEG Packet Description> CODE OBJECT ... ... code_obj_info.flt FILTER #include pat_info.flt <When PIDS=NAS OOB> #include pmt_info.flt FILTER <Output HITS time est.> #include cat_info.flt #ifdef SHOW_CODE_OBJ END FILTER #include code_obj_info.flt #include code_obj_pids.def ... ... <When PID = OBJ1> #include oob_ip.flt END FILTER <Output information> ... ... ... END FRAME END FRAME #endif END FILTER Figure 4 - Representing Encapsulation Data Description became very useful when analyzing F-16 radar data. Each of the fields within a frame must be APPLYING MACQ TO MPEG DATA defined. The data description block within the filter identifies fields of data, such as the packet sync (47 Hex), the continuity counter, DVA Group began a research program in or the PID fields of an MPEG packet. 2002 known as “Crown Royal” or CR to identify whether MAcq could be used to parse MPEG data and generate text output files. Variable Length Data Processing Frames of Data While MPEG packets are fixed format (188 bytes or 204 bytes), UDP / IP data is not. The length, however, can be readily Figure 4 shows how MAcq filter files can determined from the contents of the UDP be used to describe and process the NAS packet.