DFRWS APAC 2021 Author Preprint
Your Car is Recording: Metadata-driven Dashcam Analysis System
Kukheon Leea, Jong-Hyun Choia, Jungheum Parka,∗, Sangjin Leea
aInstitute of Cyber Security and Privacy (ICSP), Korea University, 145 Anam-ro, Seongbuk-Gu, Seoul, South Korea
Abstract Dashcam as an on-board camera is useful as a source of potential digital evidence not only to reveal the truth of a traffic accident but also to explain the situation of a crime scene as a moving surveillance camera. It stores multimedia data as well as a variety of additional information needed for accident investigation including time, location, speed, accelerometer, etc. Under these circum- stances, various studies have been conducted for dashcam forensics, but most of them focused mainly on extracting and interpreting visual video frames. In this paper, we identify and classify various metadata generated by 14 dashcam models produced by 11 manufacturers. Furthermore, we develop a normalized database schema to manage different multiple metadata and then discuss several dashcam forensic activities based on it. In addition, a prototype open-source tool is presented to support the proposed metadata-driven dashcam forensics. Keywords: Dashcam, onBoardCam, Metadata, Container, Footage, Digital forensics
1. Introduction to issues such as bankruptcy of manufacturers and corruption of files having proprietary formats. Under these circumstances, Dashboard camera (Dashcam) is an embedded recording de- various studies have been conducted for dashcam forensics, but vice that is mounted on a vehicle, in order to record driving and most of them focused mainly on extracting, recovering and an- parking situations as well as store them as multimedia container alyzing visual frames. In other words, there is a lack of sys- files. It has become an essential element of vehicles according tematic research on non-visual metadata that can be stored in a to changes of related laws and regulations. For example, in U.S., standard or non-standard manner. commercial vehicles such as police cars, buses and taxis install To respond this situation, we first identify and classify var- it to comply with Commercial Motor Vehicle Safety, and pri- ious metadata within 14 dashcam devices from the perspec- vate vehicles including Uber and Lyft use it for reasons like tive of different levels of data management, including filesys- preparing an evidence of accident investigation and resolving tem and application. This step includes reverse-engineering the disputes on insurance claims [1, 2, 3]. Due to a considerable storing structure as well as interpreting the meaning of each rise in the number of accidents and thefts, market analysts pre- metadata. The systematized result of metadata as a high-level dict that the global dashcam market is expected to register a overview of dashcam forensics could be used as a reference CAGR (Compound Annual Growth Rate) of 15.4% during the in analyzing unknown dashcams. Next, we store the identified forecast period of 2019-2024, starting with USD 2,519 billion metadata in a normalized form, and then show that integrated in 2018 [4]. metadata databases can support digital forensic activities. As In general, dashcam stores proprietary multimedia data in a examples of these activities, we discuss ‘event reconstruction’ removable storage device such as SD Card, and major manu- based on a cross-metadata timeline generation, ‘abnormal de- facturers usually provide a dedicated viewer to enable exam- tection’ through correlating multiple timestamps, and ‘source iners to analyze visual data along with non-visual information identification’ through extraction of unique characteristics from including time, location, speed, etc. Although several forensic a set of metadata. Then, to demonstrate our proposal, we de- tools provide advanced analysis functions especially for mul- velop and open-source parsers and plugins for plaso [5] that can timedia data, they are usually unable to thoroughly process a process dashcam related data including well-known multimedia variety of dashcam models from multiple manufacturers. That formats, NMEA (National Marine Electronics Association) [6] is, though examiners rely on dedicated viewers from manufac- records and customized proprietary data formats. tures for analysis of additional information (metadata) pertain- In summary, this paper makes the following contributions: ing to each video frame, they may have difficulty in responding • Identifying metadata generated by 14 dashcams produced by 11 manufacturers ∗Corresponding author. Email addresses: [email protected] (Kukheon Lee), • Proposing a normalized database schema to store and man- [email protected] (Jong-Hyun Choi), age multiple different metadata [email protected] (Jungheum Park ), [email protected] (Sangjin Lee) • Introducing dashcam forensic activities using integrated URL: http://forensic.korea.ac.kr (Sangjin Lee) metadata databases
Preprint submitted to DFRWS APAC 2020 July 3, 2020 DFRWS APAC 2021 Author Preprint
• Demonstrating the proposed concept through developing 2.1.3. Video codecs an open-source tool based on plaso Digital video devices use a codec to compress video data efficiently. The popular standards of codec for dashcmas are The remainder of this paper is structured as follows: Sec- H.264 [9] and H.265 [10]. For efficient compression, a video tion 2 summarizes backgrounds and existing studies relating data stream consists of a set of GOPs (group of pictures). As to dashcam forensics. Section 3 describes target devices and an example of H.264, there are several types of components as experimental environments, and then Section 4 identifies and illustrated in Figure 1: SPS (Sequence Parameter Set) and PPS interprets various metadata generated by dashcams. Section 5 (Picture Parameter Set) that include decoding information, IDR proposes a strategy for integrating and utilizing those metadata. (Instantaneous Decoding Refresh) that can be decoded indepen- Section 6 develops an open-source tool to prove our proposals. dently, and Non-IDR that needs to reference adjacent frames to Finally, Section 7 present conclusions and future works. be decoded as a picture. These components manage data using NAL (Network Abstraction Layer) and RBSP (Raw Byte Se- quence Payload) structures [11]. 2. Background and Related Works
GOP 2.1. Background IDR Non-IDR Non-IDR Non-IDR SPS PPS ··· 2.1.1. Basic operations of dashcam Picture Picture Picture Picture
Data storage. Dashcams mainly use flash memory based stor- Decode info. Reference NAL + RBSP age devices such as SD Card and Micro SD. Therefore, a stor- age device used in a dashcam has a relatively small capacity Figure 1: Simplified internal structure of an H.264 encoded data stream compared to one used in a host system like PC. In addition, it is not hard for ordinary users to access stored multimedia data, 2.2. Related Works due to the characteristic as a removable storage device. For ref- Although the previous studies on dashcams and multimedia erence, it should be noted that there also exist cloud-connected files have been focused mainly on how to recover deleted or dashcams that can upload multimedia files to linked online ser- damaged video data [12, 13], there are several studies on ana- vices, and it would be one of our future works. lyzing multimedia container files for dashcam forensics. Gloe et al. studied the structural characteristics of multime- File management. While a dashcam is operating, there may oc- dia containers [14]. Their goal was source identification, dis- cur a situation where its storage device is full, due to 24/7 data tinguishing between original and post-processed videos, using generation. In this situation, dashcams usually delete the oldest the feature. They analyzed 19 digital camera models, 14 mo- multimedia file before creating a new one. bile phone models, and 6 video editing toolboxes. They used the following two features: (1) chunk hierarchy (sequence) in Recording mode. Dashcams support various recording modes, container files from each device, and (2) signature values that and each mode is used as a condition to begin recording at a spe- can be stored in special areas such as INFO chunk in AVI and cific time. Basically, there are two modes, normal (= driving) non-standard chunks in MP4. and parking based on whether a vehicle is started or not. There Song et al. proposed an algorithm for tampering detection is also event mode for recording when a shock is detected from using the internal structure of a video file [15]. They utilized 3-axis accelerometer sensor. In addition, manual mode can be chunk sequences generated by several video editing programs. used to begin recording manually whenever users need it. De RosaFigure et al. proposed 2: Simplified a methodology internal for content-based structure of an H.264 encoded data stream | h264 multimedia forensics [16]. Their methodology consists of four 2.1.2. Structure of multimedia container files steps: (1) metadata extraction and analysis, (2) audio-visual in- A multimedia container format is used to store multiple ob- spection, (3) results analysis, and (4) reporting. Although a mul- jects (e.g., video, audio and text) in a single file. The MP4 (ISO timedia forensic guideline was provided, it has limitations be- Based Media File Format) [7] and AVI (Resource Interchange cause it was based on only EXIF data. File Format) [8] are the most popular file formats to handle An and Lee presented an analysis method for non-image data multimedia data. Both formats store multiple types of data us- for 6 dashcam models [17]. They proposed the following 5-step ing a specific unit such as chunk in AVI and box (or atom) in procedure: (1) analysis on video data structures, (2) extraction MP4. From here, we will use the term chunk to indicate a ba- and reconstruction of video streams, (3) analysis on non-visual sic structure inside AVI and MP4 container files. In general, data structures, (4) extraction and reconstruction of non-visual multiple chunks have a hierarchical structure for managing dif- information, and (5) relevance analysis between visual (video) ferent types of data, and each chunk consists of type, size and and non-visual information. This work has a limitation that they data fields. It should be note that there are pre-defined standard- just proposed a concept without detailed explanation. ized chunks, but developers can define their own chunks to store Cha et al. suggested requirements (operational functions) that their own data. Due to this characteristic, different hardware or a dashcam forensic tool should meet [18]. Through analyzing software may create various unknown chunks having unknown 10 dashcam models, they derived requirements for analysis, data structures. management, video playback, and recovery functions. 2 DFRWS APAC 2021 Author Preprint
Table 1: List of dashcam devices for experiments levels of data management, including filesystem, operating sys- Manufacture Model Filesystem Container Origin Release tem and application. To represent the identified metadata, we Anycom Thepoint FAT32 AVI KR 2015 define 7 main-levels (Level-1) and 21 sub-levels (Level-2) as BlackVue DR4500L-FHD FAT32 MP4 KR 2016 Dabonda DBL-1000H FAT32 MP4 KR 2014 shown in Table 2. In the following subsections, each metadata Dabonda DBH-4000H FAT32 AVI KR 2014 is represented using the defined abbreviation (Lv1 or Lv1Lv2). FineVu CR-500HD FAT32 AVI KR 2013 For reference, Appendix Table A2 provides an integrated view iRoad T-35 FAT32 AVI KR 2013 iTronics ITB-100SPW FAT32 MP4 KR 2013 of the metadata found in this study. LUKAS LK-7950WD FAT32 AVI KR 2015 Mercedes-benz Starview FAT32 AVI - 2018 Table 2: Taxonomy of dashcam metadata and their abbreviations Thinkware FXD900 FAT32 AVI KR 2013 Thinkware QXD3000 FAT32 MP4 KR 2018 Level1 Level2 Description Thinkware Z500Plus FAT32 MP4 KR 2019 cn chunk name TOUCHGO K7 FAT32 AVI KR 2018 Container (C) Transcend DrivePro220 FAT32/exFAT MP4 TW 2015 cs chunk sequence ch channel Device (D) mo model ve version Harjinder studied about the admissibility of the dashcam ev- Location (L) - location idence submitted online and the metadata from dashcams [19]. ev event (when shock) ma manual This study summarizes the techniques to consider when submit- Recording mode (R) ting dashcam evidence online. Also, 7 dashcams were briefly no normal (driving) analyzed to review the stored metadata. However, this study an- pa parking gs g-sensor alyzed only results from several tools. As a result, the authors Status (S) sp speed did not provide any details on internal structures of dashcam vo voltage data. da date accessed dc date created As a result of literature review, we found that there has been Time (T) no study in terms of thorough analysis of dashcam related meta- tc time created (incl. date) tm time modified (incl. date) data (i.e., non-visual information) to support practical forensic fs frame per second activities. ln language User preference (U) pl profiles and levels re resolution (width and height) 3. Experimental Environment tz timezone
To select dashcam devices for experiments, this work con- sidered several factors including release date, supported filesys- 4.2. Filesystem-related metadata tems, supported multimedia file formats, recording modes, num- As target models support well-known FAT32 and/or exFAT ber of camera channels, and watermarked text. As a result, 14 filesystems, it is possible to get useful metadata from directory dashcams from 11 manufacturers were selected, and Table 1 entries of FAT. shows summarized information on them. After selecting target devices, each device was mounted on 4.2.1. Directory hierarchy a vehicle to generate various events including ‘driving’ and As shown in Appendix Table A1, the directory hierarchy ‘parking’, in order to create experimental datasets. During the where multimedia container files are stored allows examiners process, we also considered more than two channels for some to identify specific recording modes (R) and timestamps (T) models. As shown in Table 1, 13 of 14 devices only support relating to the container files. Specifically, 12 models except FAT32, but Transcend’s DrivePro220 allows users to choose Starview and ITB-100SPW use recording modes (R) for nam- either FAT32 (default) or exFAT. ing top-level directories, and 2 models (DBH-4000H and T-35) record creation dates (Tdc) of container files in sub-directory names. As each device generally creates a unique directory hier- 4. Dashcam Metadata Internals archy, it is necessary to obtain device-specific information prior A dashcam continuously generates and stores various types to dashcam forensics. of data depending on conditions of a vehicle. This section iden- 4.2.2. File naming rules tifies multiple metadata that can be found in dashcam’s storage devices and describes their meaning for forensic activities. Under each of the directories identified above, there are basi- cally multimedia container files, and each dashcam model uses a set of rules to assign names to the files. Therefore, through the 4.1. Summary of findings name of a specific multimedia file, it is possible to get useful Figure 2 shows the proposed metadata-driven dashcam metadata relating to the file, including channel (Dch), recording data analysis system. We identified 22 meaningful information modes (R), time created (Ttc), etc. (metadata) in various locations, through a systematic analysis Different models have different rules for file naming, as sum- of dashcam’s storage devices from the perspective of different marized in Appendix Table A1. Examiners can obtain R, T and 3 DFRWS APAC 2021 Author Preprint
Dashcam Storage Filesystem Analysis Integrated Metadata Analysis
Allocated Area Start Directory Name File Name cn chunk name C R·T R·T·D Unallocated Dashcam related Dashcam cs chunk seq. local local Integrated Area Data Carving Analysis Result Signature mo model Directory / File Timestamp Database (C·D·L·S·T·U) Database D ve version T ch channel local Carving (MP4, AVI, NMEA, etc.) L lo location
ev event OS/Application File Analysis ma manual R no normal Multimedia (Video) File Other Files
pa parking Allocated Chunks Unused Area Location & gs g-sensor Status Info. Source List of Chunks Meta Chunks Identification S sp speed Free NMEA (TEXT) (Verification) C D·T·U Chunks vo voltage local L·S·T Stream Chunks time UTC tc created D·R·T time local or UTC Event tm Video Stream Video Stream Custom Binary modified … Subtitle Stream T (Ch 1) (Ch N) Reconstruction date Unused Bytes da L·S·T L·S·T accessed after the Last local or UTC date local or UTC dc Valid Chunk created Decoding SEI NAL IDR/Non-IDR System log Abnormal detection frame per NAL NAL fs based on second R·T U L·S·T·U multi-timestamps ln language local or UTC local or UTC profile Decoded Video Frame Carving U pl (including Watermarks) User Preferences level (Chunk, Frame, etc.) re resolution C·D·L·S·T·U D·T·U local local or UTC Metadata-driven Dashcam Data Analysis tz timezone
Figure 2: Metadata-driven dashcam data analysis (our findings and proposals)
D through file names generated by 13 devices except Drive- 4.3. OS/Application-related metadata Pro220. For example, the following is the name assigned to a Various types of metadata can also be found from operating multimedia file from DR4500L-FHD: system and application-related files. 2019-11-09-13-45-32 F event.mp4 4.3.1. Container files: list of chunks As shown in the name, ‘YYYY-MM-DD-hh-mm-ss’ means Multimedia container formats generally store multiple types creation time (Ttc), which is usually the same as the creation of data using a specific unit such as chunk. Thus, a multime- timestamp value stored in an associated directory entry of FAT dia container file may have unique chunk names (Ccn) as well filesystem. The next ‘F’ stands for one of camera channels as a chunk sequence (Ccs), according to the software to gener- that are distinguished by using characters such as S (Single), ate it [15]. That is, there is a possibility to identify a specific F (Front), R (Rear) and D (Dual). The last ‘event’ indicates a dashcam model or multimedia editing tool through those chunk certain condition for recording that can be one of event (when related information. An example in Figure 3 shows two lists shock), manual, normal (driving) and parking. In summary, of chunks generated by two different devices. As shown in the the name given as an example means that it has video data figure, container files from T-35 and LK-7950WD have clearly of the front camera when a shock was delivered at 2019/11/09 different chunk hierarchies. 13:45:32 (Local). Our experiments revealed that all models supporting the MP4 format generate unique chunk sequences. However, in the case 4.2.3. Other considerations of several models (including Thepoint, FXD900 and K7) that In addition to directory/file names, ‘timestamps’ (including only support the AVI format, it is difficult to distinguish be- created, accessed and modified) stored in directory entries of tween individual models by using only a list of chunks, because FAT filesystem will of course be useful for forensic purposes. they create an identical chunk sequence. The details of chunk For instance, a creation timestamp can be used as a factor to names and sequences found in the remainder 12 dashcam mod- determine whether a file was normally created by a dashcam or els are shown in Appendix Figure A1. copied from an external system. Moreover, due to the nature of 24/7 operation, multimedia 4.3.2. Container files: meta chunks files are frequently created and deleted in a storage device. There are special-purpose chunks that contain meaningful in- So, examiners need to consider special recovery (carving) pro- formation from a digital forensic perspective. In particular, as cesses for residual data (i.e., deleted directory entries or oper- depicted in Figure 4, it is important to note that meta chunks ating system/application-related files) within unallocated areas may include timestamps, resolution, number of frames, dura- of the storage device. tion of video/audio data, device information, etc. 4 DFRWS APAC 2021 Author Preprint
RIFF-AVI RIFF-AVI information in an avc1 (H.264) or hvc1 (H.265) chunk within a LIST – hdrl LIST – hdrl stsd (sample description) chunk. avih avih Interestingly, CR-500HD generates SEI (Supplemental En- LIST – strl LIST – strl strh, strf strh, strf, JUNK hancement Information) NALs [9], and they are used to store LIST – strl LIST – strl metadata such as time (T), location (L) and status (S ) in a pro- strh, strf strh, strf, JUNK prietary unknown format defined by its manufacturer. For your LIST – strl LIST – strl guidance, the result of reverse-engineering its storing structure strh, strf strh, strf, JUNK LIST – strl JUNK is as shown in Figure 5. strh, strf LIST – INFO LIST – movi ISFT [00dc, 01dc, 02wb, 03tx ...] JUNK 4.3.4. Container files: decoded video frame idx1 LIST – movi JUNK [00dc, 01wb, 02sb ...] It is possible to obtain video frames (a set of still images) by idx1 decoding a set of IDR and Non-IDR NALs within the above
(a) T-35 (b) LK-7950WD mentioned video stream. As a result of analysis on the decoded video frames, we confirmed that all 14 models insert informa- Figure 3: List of chunks created by two dashcam models tion, including time (T), status (S ), device (D) and user prefer- ftyp RIFF-AVI RIFF-AVI ftyp ence (U), into each frame as a watermark. moov LIST - hdrl ... moov Figure 6 summarizes types of watermarks found in target mvhd (푇푡푐,푡푚, 푈푓푠, duration) avih (푈푟푒,number of frame) LIST – INFO mvhd (푇푡푐, 푇푡푚) trak LIST - strl models. Although positions vary, it can be seen that the compo- INAM (name) trak strh (푈 ,duration) tkhd (푇푡푐,푡푚,푈푟푒,duration) 푓푠,푟푒 sition of stored metadata is similar across different models. To ICMT (comments) tkhd (푇푡푐, 푇푡푚) mdia FigureLIST 3: –ListINFO of chunks created by two dashcam models ISFT (software) mdia effectively utilize this information for forensic activities, exam- mdhd (푇푡푐,푡푚) INAM (name) ICRD (푇푡푐) mdhd (푇푡푐, 푇푡푚) iners can try to extract text from each frame by using OCR (Op- ... ICMT (comments) ...... ISFT (software) tical Character Recognition) engines such as Tesseract [11, 20]...... ICRD (푇푡푐) (b) MP4–meta chunks ...
(a) MP4–meta chunks (b) AVI–meta chunks 4.3.5. Container files: subtitle stream
Figure 4: Meta chunks stored in MP4 and AVI container files Most of the target models used in our experiments create a customized ‘subtitle’ stream to store metadata such as time (T), From MP4 containers, it is possible to identify various meta location (L), and status (S ). Since it does not follow the standard chunks including mvhd (movie header), tkhd (track header) and subtitle formats, typical media players are unable to properly interpret the customized subtitle data. mdhd (media header), and they commonly contain created (Ttc) and modified (Ttm) timestamps in UNIX time format. The Ttc For example, MP4 files created by several models including and Ttm mentioned here are usually the same, which allows ex- DR4500L-FHD have an independent track for text-type data to aminers to understand when an MP4 container was first created. store additional information in their own format. In addition, avc 까지? 논문In addition,전체 체크하고 regarding수정하기 AVI, there are some representative meta in the case of AVI files created by several models such as T- chunks including avih (avi header), strh (stream header) and 35, there exists a subtitle chunk with names like ##tx, ##sb and Figure 4: Meta chunks stored in MP4 sub-chunksand AVI container of INFO list files chunk (e.g.,| meta_chunks.pdfICRD, INAM and IPRD). ##st within a movi list chunk, as shown in Figure 7. As men- For example, an INAM chunk generated by CR-500HD contains tioned above, ## means a sequence number assigned to each ‘CR-500HD’ as its model name (Dmo) in ASCII string format. stream. So, if there are two video streams (00 for Front and 01 For reference, it should be noted that the INFO list chunk for Rear) along with one audio stream (02), the lastly stored may not exist in some AVI containers, because it is not manda- subtitle stream will be assigned as ‘03’ as shown in the figure. tory in the format specification. As a result of analysis, 11 of 14 models have a customized ‘subtitle’ stream, and 10 of them store metadata (L, S gs and so 4.3.3. Container files: video stream on) in the standardized NMEA format. The other model, DBL- A video stream of a multimedia container file is an area in 1000H, that does not contain a GPS module records S gs in a which a set of GOPs generated during a predetermined time customized binary format. period (usually 1 to 3 minutes) is stored. In other words, it is an area in which raw data containing actual visible information is Compressed video stream h.264 start word SEI Header saved, but some useful metadata can also be identified here. The H.264 and H.265 codecs basically store information 0000h 0 0 d c chunk_size 00 00 00 01 06 Unknown needed for decoding such as frames per second and resolution 0010hY Y Y Y M M D D h h m m s s0000 in SPS NAL [11]. Regarding AVI, raw data (including decod- 0020h latitude (double) longitude (double) ing information and a set of GOPs) generated by a codec is 0030h speed (double) Unknown stored as it is in ##dc (compressed video stream; ## means a 0040h g-sensor x value (ASCII) g-sensor y value (ASCII) sequence number assigned to each stream) chunks under a movi 0050h g-sensor z value (ASCII) . . . list chunk. However, the MP4 format separately stores decoding Figure 5: A proprietary SEI NAL data structure created by CR-500HD 5
Figure 5: A proprietary SEI NAL data structure created by CR-500HD | SEI.pdf DFRWS APAC 2021 Author Preprint
watermark position number on footage model [position: list of watermark] TS_N####.MP4 $GPRMC Format Thepoint [3:푇 ,S ,퐷 ] 푡푐 gs 푣푒 pair $GPRMC,050015.00,A,3734.612168,N,12701.649440,E,18.9, DR4500L-FHD [3:퐷 ,퐷 ,푇 ],[4:푆 ] TS_N####.NMEA 푚표 푣푒 푡푐 푣표 68.7,060819,6.1,W,A*29 2 1 DBL-1000H [3:푆 ,푈 ],[4:푇 ] 푔푠 푓푠 푡푐 GPRMC Sentence ID - DBH-4000H [3:퐷푚표,푇푡푐] 050015.00 UTC TIME 05:00:15.00(UTC) CR-500HD [3:푆푣표,푆푔푠],[4:푈푓푠,푆sp,푇푡푐] $GSENSOR,-216,-1 ... A Status valid T-35 [3:퐷푣푒] 3734.612168 Latitude 37°34.612168’ ITB-100SPW [2:푇푡푐] $GPRMC,050015.00,... LK-7950WD [3:푇푡푐,푆sp],[4:푆푣표] N N/S Indicator North $GPVTG, ,T,,M,... Starview [3:푇푡푐,Sgs,푣표,퐷푣푒,푆sp] 12701.649440 Longitude 127°01.64944’ FXD900 [3:푆 ],[4:푇 ] E E/W Indicator East 3 4 푣표 푡푐 ... QXD3000 [3:퐷푚표,푣푒,푆푣표,푇푡푐,푆sp] 18.9 Speed(knots) 18.9 knots(35km/h) Z500Plus [3:퐷푚표,푣푒,푆푣표, 푇푡푐,푆sp] $GPSGSV,3,1,12,03... 68.7 Course over ground from North K7 [3:푇푡푐,Sgs,푠푝,푣표,퐷푣푒] 060819 UTC Date 2019/08/06 [3:퐷 ,퐷 ,푆 ,푇 ,푆 ] 푚표 푣푒 푣표 푡푐 sp DrivePro220 [3:퐷푚표,푇푡푐] $GPGLL,3734.61216... 6.1 Magnetic variation - W deg West $GPGGA,050015.00... Figure 6: Watermarks inserted in video frames A*29 checksum -
Figure 8: An example of .NMEA file created by DrivePro220 4.3.6. Container files: unused area rec_YYYYMMDD_hhmmss_.MP4 pair Unused areas may exist inside a multimedia container file. rec_YYYYMMDD_hhmmss_.dat Examples of the areas include the JUNK chunk of AVI we well utc 0 - as free and skip chunks of MP4, which can be used for padding. SENS size 0123456789ABCDEF Figure 6: Watermarks inserted in video frames | watermark.pdfrecord 1 0000h unix time z-value y-value x-value These chunks are also used for special purposes, and there have record 2 0010h speed ...... been cases where device information (D) and creation times- record 3 (SENS record : G-sensor data) tamp (Ttc) are stored [14]. Among the target devices tested here, Figurerecord n8: An example of .NMEA file created by DrivePro220 | NMEA_GPRMC.pdf T-35 Starview ... utc 0 bool an AVI file produced by or has an interesting GPSI size 0123456789ABCDEF characteristic of storing its file name in a JUNK chunk. record 1 0000h unix time on road latitude longitude Additionally, container files with a ‘fixed’ length may have record 2 record 3 0010haltitude ...... unused areas outside their valid ranges [11]. For example, an record n (GPSI record : location data) MP4 file from QXD3000 utilizes the lastly located free chunk to manage the remainder of the valid data. Figure 9: An example of .DAT file created by ITB-100SPW While these unused areas are not related to the playback of multimedia files, they are likely to include parts of meaningful 10 shows a part excerpted from a system log, having multiple data (e.g., meta chunks, video streams, subtitle streams, etc.) event entries such as power on/off, storage device usage, file that has been generated in the past. Therefore, it needs to be creation and so on. confirmed by applying ‘carving’ techniques specially designed for dashcam related data. User preferences.FigureAdditional 9: An example metadata of .DAT can alsofile becreated identified by ITB-100 | dat_file.pdf from files where user preferences (configurations) are stored. 4.3.7. Other files As a result of analyzing the target models, we confirmed that dashcams generally use standard formats like SQLite and XML Location and status information. Dashcam devices also gen- for managing device’s HW/SW information, video quality, au- erate other files than multimedia container files. Here, we in- dio volume, screen brightness, language, etc. As an example, troduce (1) a text file (.NMEA) where NMEA information is Figure 11 indicates an XML-based file from T-35. stored and (2) a custom data file (.DAT) specially defined by a manufacture. First, a .NMEA file is created to record additional information about video frames for each multimedia container 5. Integration and Analysis of Dashcam Metadata file. As shown in Figure 8, it allows examiners to identify lo- cation (L), time (T), and status (S ) through interpreting text- This section defines a database schema for integrating dash- based standard NMEA messages. Next, customized data struc- cam metadata introduced in the previous section, and discusses tures from manufactures are also used rather than standardized its usefulness through three examples of digital forensic activi- formats, like indicated in Figure 9. Our reverse-engineering re- ties: event reconstruction, abnormal detection and source iden- sults revealed that .DAT files in ITB-100SPW manage a set of tification. location (L) and status (S gs,sp) values by using two proprietary data structures, SENS and GPSI. 5.1. Integration To efficiently store and manage dashcam metadata, we define System logs. Some models including T-35 record detailed op- a database schema as illustrated in Figure 12. erational events in a text-based log file. As an example, Figure In this schema, ‘device info’ is a table for managing basic information (e.g., manufacture, model and version) on dash-
subtitle stream Chunk size ‘gsensor’ cam devices. Tables used for directories (‘directory info’) and files (‘file info’) in a storage device is composed of fields for 0000h 30 33 74 78 70 00 00 00 67 73 65 6E 73 6F 72 2C 03txp...gsensor, information from both normal and recovered entries. In addi- 0010h 2D 31 38 2C 33 38 2C 2D 31 32 33 00 00 00 00 00 -18,38,-123..... tion, metadata (‘metadata’) and time (‘time value’) tables have
x:-18 y:38 z:-123 data type and data source fields, in order to manage various metadata introduced in this paper. Figure 7: An example of g-sensor data structure in an AVI container file 6
Fig. structure of g-sensor data in avi file DFRWS APAC 2021 Author Preprint
[2019/11/09 09:49:46] Power On Among them, wt and gi include continuous timestamps from [2019/11/09 09:49:46] Insert MMC/SD card [2019/11/09 09:49:56] Recording Start the time a file was first created until the time it was last saved. [2019/11/09 09:50:06] Create /mnt/mmc/NORMAL/20191109_093930/20191109_095006_I2.tmp ... Especially, gi can be extracted from various sources including [2019/11/09 09:53:07] Closed /mnt/mmc/NORMAL/20191109_093930/20191109_095207_I2.avi [2019/11/09 09:59:53] ------SEI NALs, subtitle stream and other files (.NMEA and .DAT). [2019/11/09 09:59:53] Enter Parking Mode [2019/11/09 09:59:54] Closed /mnt/mmc/NORMAL/20191109_093930/20191109_095909_I2.avi The relationship between time values mentioned above is rep- ... [2019/11/09 15:24:27] Exit Parking Mode resented as follows: [2019/11/09 15:24:27] ------[2019/11/09 15:24:37] Create /mnt/mmc/NORMAL/20191109_125200/20191109_152437_I2.tmp [2019/11/09 15:24:42] Closed /mnt/mmc/NORMAL/20191109_125200/20191109_152437_I2.avi dn ≤ f n = detc = sl ≤ mctc,tm ≤ wt = gi <= detm [2019/11/09 15:24:43] Recording Stop [2019/11/09 15:25:26] Recording Start The above conditions can be used to validate distributions of Figure 10: An example of system logs managed by T-35 time values identified from a dashcam device. It should be noted that only timestamps (mc, wt, gi) inside multimedia container formats need to be considered for individual multimedia files Figure
device_info directory_info metadata time_value Dashcam signature DB device_id INT type_list device_id INT rec_id INT rec_id INT manufacture TEXT dir_id INT file_id INT file_id INT signature_table 0 unknown dir_info TEXT model TEXT data_source INT data_source INT signature_id INT 1 front dir_name TEXT version TEXT data_type INT data_type INT manufacture TEXT 2 rear dir_path TEXT filesystem TEXT std_time TEXT time_value TEXT model TEXT cam_channel INT 3 dual is_deleted BOOL data1_value TEXT utc_time INT ... version TEXT container_format INT data2_value BLOB filesystem TEXT location_type INT 10 customized 11 AVI cam_channel TEXT filenaming_rule TEXT file_info data_type_list data_src_list container_format TEXT timezone TEXT 12 MP4 ... file_id INT 10 container 10 filesystem filenaming_rule TEXT language TEXT list_of_directory TEXT 20 NMEA dir_id INT 11 chunk_name 11 directory_name 21 TEXT(ASCII) file_name TEXT 12 chunk_seq. 12 filename list_of_chunks TEXT 22 TEXT(unicode) file_type INT ...... sps_nal TEXT ... is_deleted BOOL 20 channel 21 list_of_chunks codec_info TEXT 30 text is_recover BOOL 21 model 22 meta_chunks custom_chunks TEXT 31 multimedia is_rawdata BOOL 22 version 23 video_stream watermark_in_frame TEXT 32 other_file rawdata BLOB ...... file_path TEXT Table 2 Figure 2
Figure 12: A proposed database schema designed to support metadata-driven dashcam data analysis
No Time Timezone source type data introduced possible forensic activities using integrated dash- 1 2020/01/05 14:20:30 UTC filesystem time created - cam metadata databases. Then, based on these findings, this pa- 2 2020/01/05 14:20:30 UTC meta_chunks time created - 3 2020/01/05 14:20:30 UTC filename record.mode normal per developed an open-source tool to demonstrate the proposed 4 2020/01/05 14:20:31 UTC subtitle location 37.583230, 127.027490 metadata-driven dashcam forensic analysis. schema.pdf Figure5 2020/01/05 12: 14:20:31 A proposedUTC subtitle databasestatus schema0.5, -0.4,designed-0.6 to supportSince the metadata current version-driven of plaso dashcamdoes notdata have analysis the ability 6 2020/01/05 14:20:31 UTC subtitle speed 14 to process multimedia files, we expect the results of this study n ... n+1 2020/01/05 14:25:10 UTC filename record.mode event to be useful in the digital forensic community. Consequentially, n+2 2020/01/05 14:25:10 UTC subtitle location 37.583099, 127.027152 the proposed metadata-driven approach will not only enhance traditional visual data-oriented analysis but can also be utilized Figure 13: An example of event reconstruction for a dashcam as a foundation for dashcam forensics in new perspectives. There are several plans for enhancing the proposed concept, such as investigating additional dashcam models from diverse files and extracting metadata by using plaso. Because the cur- manufacturers, identifying further metadata from them, deriv- rent version of plaso does not have the ability to handle multi- ing an efficient metadata combination for source identification, Figure 13:media An example data, of developing event reconstruction new parsers for and a pluginsdashcam is| event_reconstruct.pdf required to and developing a stable tool for supporting efficient dashcam support this part. Specifically, it should be able to process stan- forensics. dard multimedia formats (i.e., AVI and MP4), NMEA records and customized proprietary data formats, as introduced in Sec- tion 4. Next, the second part is to perform filtering, interpret- Acknowledgement ing and integrating dashcam-related records from an output of plaso. As a result of these analysis operations, a normalized This work was supported by Institute of Information & Com- result database is created, as well as an existing dashcam signa- munications Technology Planning & Evaluation(IITP) grant ture database is updated if necessary. funded by the Korea government(MSIT) (No.2018-0-01000, Development of Digital Forensic Integration Platform) 6.2. Prototype implementation We are implementing the design concept described above as References an integrated automated tool. To demonstrate our proposal, the current version of a Python-based prototype is released as an [1] UBER, Can I use a video camera (2019 (accessed January 05, 2020)). 1 URL https : / / help . uber . com / partners / article / can - i - open source project . use- a- video- camera- ?nodeId=efaad152- cbb6- 45fe- 9d7d- 911842d21c8b [2] Lyft, Recording device policy (2019 (accessed November 20, 2019)). 7. Conclusion and future work URL https : / / help . lyft . com / hc / en - us / articles / 115012923127-Safety-policies [3] D. Stitilis,ˇ M. Laurinaitis, Legal regulation of the use of dashboard cam- This paper identified various types of metadata within 14 eras: Aspects of privacy protection, Computer Law & Security Review dashcam models from the perspective of different levels of data 32 (2) (2016) 316–326. management. This work also proposed a normalized database [4] M. Intelligence, Dashboard camera market - growth, trends, and forecast schema to store and manage multiple different metadata, and (2019 - 2024), Tech. rep., Mordor Intelligence (2019). [5] K. Gudjonsson, plaso ((accessed March 05, 2020)). URL https://github.com/log2timeline/plaso [6] N. M. E. Association, et al., NMEA 0183–Standard for interfacing marine 1[GitHub URL] https://github.com/kukheon1109/Dashcam Plaso electronic devices, NMEA, 2002. 8 Preprocessing Analysis Result Plaso for Dashcam Parser for Parser for Integrated Analysis Dashcam Filtering Interpretation Result Database Signature Database AVI file MP4 file
Parser for Parser for Database Integration NMEA records custom binary controller DFRWS APAC 2021 Author Preprint
Preprocessing Analysis Result Parser for Parser for Integrated Analysis Dashcam Filtering Interpretation AVI file MP4 file Result Database Signature Database
Parser for Parser for Database Integration NMEA records custom binary controller Plaso for dashcam
Figure 14: A design concept for implementation
[7] ISO, Iso/iec 14496-12, information technology — coding of audio-visual objects — part 12: Iso base media file format (2020). [8] Microsoft, AVI RIFF File ReferenceFigure (2018 14: (accessed A design May 3, 2018)).concept for implementation | plaso.pdf URL https://docs.microsoft.com/en- us/windows/win32/ directshow/avi-riff-file-reference [9] T. ITU, Advanced video coding for generic audiovisual services, ITU-T Recommendation H. 264. [10] T. ITU, H.265: Infrastructure of audiovisual services – coding of moving video – high efficiency video coding, ITU-T Recommendation H. 265. [11] J. Park, S. Lee, Data fragment forensics for embedded dvr systems, Digi- tal Investigation 11 (3) (2014) 187–200. [12] K. Alghafli, C. Y. Yeun, E. Damiani, Techniques for measuring the prob- ability of adjacency between carved video fragments: The vidcarve ap- proach, IEEE Transactions on Sustainable Computing. [13] G.-H. Na, K.-S. Shim, K.-W. Moon, S. G. Kong, E.-S. Kim, J. Lee, Frame- based recovery of corrupted video files using video codec specifications, IEEE Transactions on Image Processing 23 (2) (2013) 517–526. [14] T. Gloe, A. Fischer, M. Kirchner, Forensic analysis of video file formats, Digital Investigation 11 (2014) S68–S76. [15] J. Song, K. Lee, W. Y. Lee, H. Lee, Integrity verification of the ordered data structures in manipulated video content, Digital Investigation 18 (2016) 1–7. [16] T. Gloe, Forensic analysis of ordered data structures on the example of jpeg files, in: 2012 IEEE International Workshop on Information Foren- sics and Security (WIFS), IEEE, 2012, pp. 139–144. [17] H. An, S. Lee, The analysis of data structure to digital forensic of dash- board camera, Journal of the Korea Institute of Information Security and Cryptology 25 (6) (2015) 1495–1502. [18] I. H. Cha, K. H. Lee, S. J. Lee, Design and implementation of car black- box forensic analysis tool through the analysis of data structure, KIPS Transactions on Computer and Communication Systems 5 (11) (2016) 427–438. [19] H. S. Lallie, Dashcam forensics: A preliminary analysis of 7 dashcam devices, Forensic Science International: Digital Investigation 33 (2020) 200910. [20] R. Smith, An overview of the tesseract ocr engine, in: Ninth Inter- national Conference on Document Analysis and Recognition (ICDAR 2007), Vol. 2, IEEE, 2007, pp. 629–633.
Appendix
This appendix contains extra tables and figures for reporting detailed findings.
9
LIST – strl tkhd, edts, mdia udta LIST – strl LIST – strl DFRWS strh, strf trak AMBA, free strh, strf strh, strf, JUNK LIST – strl tkhd, edts, mdia trak LIST – strl JUNK strh, strf udta tkhd, edts, mdia strh, strf LIST – INFO LIST – strl TIME, ENCY, GPSR, SENS trak LIST – strl INAM, ICMT, ISFT strh, strf free tkhd, mdia strh, strf JUNK APAC LIST – movi trak JUNK LIST – movi [00dc, 01dc, 02wb, 03tx ...] tkhd, mdia LIST – movi [00dc, 01wb ...] idx1 [00dc, 01dc, 02wb, 03tx ...] idx1 idx1 2021 10
Figure A1: List of chunks created by 12 dashcam models DFRWS APAC 2021 Author Preprint
Table A1: Directory hierarchy (including directory naming rules) in storage devices used by dashcams
Model Category Description Dir. PARKING(pa), SETTING, USER(ma), DRIVING(no), EVENT(ev) Anycom File RM YYYYMMDD hhmmss CH.avi (e.g., Rec 20191109 134532 D.avi) Thepoint Note RM: Park(pa), Manual(ma), Rec(no) or Event(ev), CH: of S(single) or D(dual) Dir. BlackBox(no), Event(ev), Parameter, Parking(pa), ParkingEvent(ev) BlackVue File YYYY-MM-DD-hh-mm-ss CH RM.mp4 (e.g., 2019-11-09-13-45-32 F normal.mp4) DR4500L-FHD Note CH: F(front) or R(rear), RM: normal(no), event(ev), motion(pa) or parking(ev) Dir. SETUP, EVENT(ev), NORMAL(no) Dabonda File YYYYMMDD hhmmss RM.mp4 (e.g., 20191109 134532 INF.mp4) DBL-1000H Note RM: EVT(ev) or INF(no) Dir. config, event(ev), normal(no) Dabonda Sub-dir. YYYYMMDD(dc) DBH-4000H File nnnnn YYYYMMDD hhmmss RM.avi (e.g., 00003 20191109 134532 N.avi) Note nnnnn: sequence number, RM: E(ev) or N(no) Dir. camcorder(ma), event(ev), normal(no), photo(ma) FineVu File YYYY-MM-DD-hh‘h’mm‘m’ss‘s’ RM.avi (e.g., 2019-11-09-13h54m32s normal.avi) CR-500HD Note RM: manual(ma), parking(pa in event dir.), normal to event(ev), event(ev), motion(pa in event dir.), normal(no) or capture(ma) Directory EVENT(ev), NORMAL(no), PARK(pa), LOG iRoad Sub-dir. YYYYMMDD(dc) T-35 File YYYYMMDD hhmmss RMCH.avi (e.g., 20191109-134532 I2.avi) Note RM: E(ev), I(no) or M(pa), CH: 1(Single) or 2(Dual) Directory Setup iTronics File RM YYYYMMDD hhmmss .mp4, RM YYYYMMDD hhmmss .dat (e.g., rec 20191109 134532 .mp4) ITB-100SPW Note RM: rec(no), evt(ev) or pmr(pa) [Note that .dat file is for storing location and status info.] Directory AlwaysMovie(no), MotionMovie(pa), EventMovie(ev), obd LUKAS File RM YYYYMMDD hhmmss CH.avi (e.g., alwa 20191109 134532 F.avi) LK-7950WD Note RM: alwa(no), motion(pa) or event(ev), CH: F(front) or R(rear) Directory root directory only Mercedes-benz File RM YYYYMMDD hhmmss CH.avi (e.g., REC 20191109 134532 F.avi) Starview Note RM: REC(no), EVT(ev), MAN(ma) or PAR(pa), CH: F(front) or R(rear) Directory driveinfo, DRIVEX, Driving(no), Driving Shock(ev), Manual(ma), Parking(pa), Parking Shock(ev), Safety Box, SETTING Thinkware File RM YYYY MM DD hh mm ss CH.avi (e.g., REC 2019 11 09 13 45 32 D.avi) FXD900 Note RM: REC(no), EVT(ev), MAN(ma), PMOT(pa) or PEVT(ev in Parking Shock dir.), CH: S(single) or D(dual) Directory driveinfo, DRIVEX, Driving(no), Driving Shock(ev), Manual(ma), Parking(pa), Parking Shock(ev), Safety Box, SETTING Thinkware File RM YYYY MM DD hh mm ss CH.mp4 (e.g., REC 2019 11 09 13 45 32 F.mp4) QXD3000 Note RM: REC(no), EVT(ev), MAN(ma), MOT(pa) or PAK(ev in Parking Shock dir.), CH: F(Front) or R(Rear) Directory driveinfo, DRIVEX, Driving(no), Driving Shock(ev), Manual(ma), Parking(pa), Parking Shock(ev), Safety Box, SETTING Thinkware File mode YYYY MM DD hh mm ss CH.mp4 (e.g., REC 2019 11 09 13 45 32 F.mp4) Z500Plus Note RM: REC(no), EVT(ev), MAN(ma), MOT(pa) or PAK(ev in Parking Shock dir.), CH: F(Front) or R(Rear) Directory CFIG, CONT(no), EVNT(ev), PARK(pa), PEVT(ev), USER(ma) TOUCHGO File RM YYYYMMDD hhmmss CH.avi (e.g., Cont 20191109 134532 D.avi) k7 Note RM: Evnt(ev), Cont(no), User(ma) or Park(pa), CH: S(single) or D(dual) Directory DCIM(ma), N-VIDEO(no), P-VIDEO(ev), SYSTEM Transcend File TS RMnnnn.mp4, TS RMnnnn.NMEA (e.g., TS N0003.mp4) DrivePro220 Note RM: D(ma), N(no) or P(ev), nnnn: sequence number [Note that .NMEA file is for storing location and status info.] ※ This table uses abbreviations for Level-2 metadata listed in Table 2. ※ RM: recording mode, CH: channel, YYYY: year, MM: month, DD: day, hh: hour, mm: minute, ss: second.
11 Table A2: Summary of dashcam metadata
Filesystem: Filesystem: Filesystem: Container: Container: Container: Model directory name file naming rule unallocated area meta chunk video stream decoded video frame2 1 Thepoint R Dch · R · Ttc ALL U f s,re U f s,pl,re Dve · Ttc · S gs DR4500L-FHD Rev,no,pa Dch · Rev,no,pa · Ttc ALL Ttc,tm · U f s,pl,re - Dmo,ve · Ttc · S vo DBL-1000H Rev,no Rev,no · Ttc ALL Ttc,tm · U f s,pl,re - S gs · U f s · Ttc DBH-4000H Rev,no · Tdc Rev,no · Ttc ALL U f s,re U f s,pl,re Dmo · Ttc DFRWS CR-500HD Rev,ma,no Dch · R · Ttc ALL Dmo,ve · U f s,re L · S gs,sp · Ttc · U f s,pl,re S vo,gs · U f s · S sp · Ttc T-35 Rev,no,pa · Tdc Dch · Rev,no,pa · Ttc ALL U f s,re U f s,pl,re Dve ITB-100SPW - Rev,no,pa · Ttc ALL Ttc,tm · U f s,pl,re - Ttc LK-7950WD Rev,no,pa Dch · Rev,no,pa · Ttc ALL U f s,re U f s,pl,re Ttc · S sp,vo Starview - Dch · R · Ttc ALL U f s,re U f s,pl,re Ttc, S gs,vo · Dve · S sp FXD900 R Dch · R · Ttc ALL U f s,re U f s,pl,re S vo · Ttc QXD3000 R Dch · R · Ttc ALL Ttc,tm · U f s,pl,re - Dmo,ve · S vo · Ttc · S sp APAC Z500Plus R Dch · R · Ttc ALL Ttc,tm · U f s,pl,re - Dmo,ve · S vo · Ttc · S sp K7 R Dch · R · Ttc ALL U f s,re U f s,pl,re Ttc · S · Dve DrivePro220 Rev,ma,no Rev,ma,no ALL Ttc,tm · U f s,pl,re - Dmo · Ttc
Container: Container: Container: Other files: Other files: Other files: 2021 3 12 Model subtitle stream unused area (AVI-junk) unused area (MP4-free,skip) location and status info. system log user preference Thepoint L · S gs,sp · Ttc - - - - Dmo,ve DR4500L-FHD L · S gs,sp · Ttc - ALL - - Dmo Author DBL-1000H L · S gs,sp · Ttc - - - - Dmo,ve · Ttm DBH-4000H L · S gs,sp · Ttc - - - - Dmo,ve CR-500HD - - - - - Dmo,ve T-35 L · S gs,sp · Ttc Dch · Rev,no,pa · Tdc - - R · Ttc Dmo ITB-100SPW - - ALL L · S gs,sp · Ttc - Dmo,ve LK-7950WD L · S gs,sp · Ttc - - - R · Ttc Dmo,ve Preprint Starview L · S gs,sp · Ttc Dch · R · Ttc - - - - FXD900 L · S gs,sp · Ttc - - - - Dmo,ve · Utz QXD3000 L · S gs,sp · Ttc - ALL - - Dmo,ve · Utz Z500Plus L · S gs,sp · Ttc - ALL - - Dmo,ve · Utz K7 L · S gs,sp · Ttc - - - R · Ttc Dmo,ve · Utz DrivePro220 - - ALL L · S gs,sp · Ttc - - ※ In this table, only Lv1 means that there exist all Level-2 metadata (i.e., R = Rev,ma,no,pa). ※ This table does not include list of chunks (C) column because all models commonly have the information. 1 ‘ALL’ means that any types of metadata can potentially exist in unallocated or unused areas. 2 Each list of metadata is presented by watermarked patterns shown in Figure 6. 3 Starview and Drivepro do not store user preference related files on external storage devices. We assume that the devices manage preference (configuration) data using internal flash memory (e.g., NAND, eMMC) rather than external SD card.