Why this document ? ... Because : Things and life today are very highly technical and scientific. Explanations may be rare or out of reach or too empirical. Words are insufficient to "print thechnical facts in mind" Acquiring knowledge is a maturation (brewing) process. Science is not inductive, rather it is coalescent. "Dans le domaine de la science, le hasard ne favorise que les esprits qui ont été préparés" PASTEUR

JPEG/MPEG Forensic knowledge base in the Court Room This document is a JPEG/MPEG "knowledge base primer" demo. It is a Mathcad 11.2a work sheet running on PC. No maths are needed as the demonstration is by images. It is presented in the general context of the admissibility of JPEG/MPEG as circumstantial proof, rather to the inadmissibilty a priori. The Fourier part (DFT, IDFT) is not arguable. The relative reliability of the JPEG/MPEG extracted from mass storage is to be submitted to experts in respect to all the technical parts of the chain. An exhaustive analysis may not be possible because each system is protected by Patent Rights (proprietary information). Therefore, this report is a guideline based on the principles and exposed as such for Judge and Counsel. Two extreme JPEG demos have been chosen for purpose. They both represent geometric forms. Example (1) is reconstructible from educated guesses, Example (2) is also geometric but totally destroyed ... So, nuance means nuance an eternal word that didn't change meaning over time. Please, read attentively... this report is personal for public use and understanding, addressing the Court Room especially.

At the end of this document you must read more and navigate http://en.wikipedia.org/wiki/JPEG http://en.wikipedia.org/wiki/Wavelet

The word is due to Morlet and Grossmann in the early 1980s. They used the French word ondelette , meaning "small wave". Soon it was transferred to English by translating "onde" into "wave", giving " wavelet ".

The Morlet wavelet is arguably the `original' wavelet. Although the discrete Haar predate Morlet's, it was only as a consequence of Morlet's work that the mathematical foundations of wavelets as a better formulation of time-frequency methods were laid. Conceptually related to windowed-Fourier analysis, the Morlet wavelet is a locally periodic wavetrain. It is obtained by taking a complex sine wave, and by localizing it with a Gaussian (bell-shaped) envelope ..... ______

Fourier: Born: 21 March 1768 in Auxerre, Bourgogne, France Died: 16 May 1830 in Paris

As the natural ideas of equality developed it was possible to conceive the sublime hope of establishing among us a free government exempt from kings and priests, and to free from this double yoke the long-usurped soil of Europe.

Joseph Fourier

Extrapolate "Fourier natural idea": "concieve/conclude the sublime hope for justice from reasoning supported by science" ? ______

What is Fourier transform ? ( jmG ) It is an approximation method (one among many more). It decomposes a function into its two trigonometric real or complex components (sinus, cosinus). There are two forms: 1. the analytical form which transforms the continuous time domain into the frquency domain 2. the discrete form for vector of values. It transforms discrete values into the spectral compositon (harmonic analysis) ... composition meaning sine/cosine components. The discrete form is universally named DFT (Discrete Fourier Transform). The "Fourier transform is maths" as such it belongs to all fields of Science, Physics, Engineering. The DFT is to advanced science and technologies what alphabet is to a language. Once in hand the DFT of some "original", very many educated manipulations can be applied for purpose, then "reverse engineered" by the IDFT (Inverse Discrete Fourier Transform), thus reconstructing a "modified original". Signal processing and Image processing would not exist without DFT/IDFT

The discrete cosine transform is the core algorithm for the JPEG image. Below, L is the (pixels) image matrix of Lena (the lady). We demonstrate the decomposition into the discrete cosine values, where the components of the highest values populate and we reconstruct the image to a degree of accuracy about equivalent to 12 decimals (so to speak for popular understanding). This is an introduction demo to the JPEG compression Part I: Image structure, compression is an unrecoverable destruction

L := L:= submatrix() L, 0 , 255 , 0 , 254 0 1 2 Ldct:= dct2d() L  max() Ldct   25202  0 137 136 133   =   1 137 136 133 Lrec:= idct2d() Ldct  min() Ldct   −3292 

The scaled recovered Lena is scale() Lrec , 0 , 255 not shown as it is the exact replicate to ± 12 d

L scale() Ldct −, 3300 , 25000 The discrete cosine transform (DCT) is an important and widely used analysis tool in signal and image processing. The DCT is particularly valuable for coding and applications, and it forms the core of the JPEG and MPEG algorithms for . Like the discrete Fourier transform (DFT), the DCT defines an orthogonal transformation which maps a signal from the time domain to the frequency domain. However, the DCT is a real-valued transform, and does not involve complex numbers. The DFT is a complex-valued transform. For a comprehensive discussion of the DCT and applications involving the DCT, refer to the book by K. R. Rao and P. Yip. We can verify the energy-compaction property of the DCT by looking at histograms of the transformed and original images, as follows:

The transformed image concentrates most of the energy in the first H:= imhist() Ldct , 256 few bins of the histogram, while the distribution of the original image is spread out. This makes the transformed image a better candidate Horig:= imhist() L, 256 for encoding or transmission, since most of the image information can be relayed with very few frequency components. For this reason k := 0 .. rows() H − 1 the DCT is used for image compression.

Comparing spectral histogram 4000

Horig k

H k 2000

0 0 50 100 150 200 250 300 k original spectral spread DCT spectral concentration

Notes • The sine transform, ST, and cosine transform, CT, of a sequence x of length N are defined as follows   j     j   ST j = sin π⋅i⋅ ⋅xi CT j = cos π⋅i⋅ ⋅xi ∑   N  ∑   N  i i • The sine transform always has first coordinate 0, so the inverse sine transform cannot recover the first element of the input x. The cosine transform contains enough information to reconstruct the original vector x completely.

Much of the information in an image can be stored in a smaller space by cosine transforming, then retaining only the coefficients which store most of the image informaiton. To do this, take a two-dimensional cosine transform by first transforming columns and transposing the result, then repeating the procedure. Example 1

()n i j p c := ()23 0 .. n 0 .. n 5 12 n, i, j define a 24 x 24 array not yet filled with values (n = odd) ... c, is a center pixel bordered Define the aperture each side square by 'p' pixels, these 'p' "border pixels" are made white by the logical definition Ai, j := (i− c < p )⋅() j− c < p of the A matrix ... Typical Mathcad compaction ! Matix A has levels 1/0 for 3D patch and bar plot 〈〉i 〈〉i T The C1, C2, C3, C4 are the two-dimensional C1 := costr() A C2:= C1 cosine transform by first transforming columns 〈〉i 〈〉i T and transposing the result, then repeating the C3 := costr() C2 C4:= C3 procedure.

A ------is a 3D patch plot, C4 a bar plot As ------is the 0,255 picture pixels C4s -----is the 0,255 picture pixels For better view of C4s, it as been magnified. It is not a picture, just a captured image of specifications: ()23 0 .. n 0 .. n 4 12 Note: to reproduce the black/white 1/0 of the Patch plot Bar plot 3D patch plot we scale the picture pixel 0..255

A C4

As:= scale() A , 0 , 255 "A" at left is Quite a different image than Lena ! C4s:= scale() C4 , 0 , 255 Still it is easy to see the concentration of information. At this point, we could reduce in two ways the volume of "bytes" to store/export an image: 1. ignore the lower diagonal As C4s 2. scaling has reduced the magnitude of the numbers thus reduced the "bytes size".

How image compression works ? By running a 10 tons truck over Lena ... NO ! We filter the uncentered transform with a mask that is nonzero on the 'p' border of the original transform area. With 'p' = 4 we've thus compressed the image content by a factor of about 3. (Of course this isn't true compression, since we're saving the remaining values to high precision; in a real compression scheme we'd quantize these values somehow.) We'll reconstruct the image from the masked transform by using the inverse cosine transform.

p := 4 p = 4

First we define a Boolean OR function:

OR( a, b ):= if() a+ b , 1 , 0 Mask i, j := OR() i< p , j< p The mask saves only transform values near the upper and left edges of the uncentered transform. Now we'll apply the mask element by element to the transform using the vectorize operator, and then reconstruct the image from the masked transform and compare it with the original image A.

→ T 〈〉i 〈〉i C4:= Mask⋅ C4 C5:= C4 C6 := icostr() C5 T 〈〉i 〈〉i C7:= C6 C8 := icostr() C7

The cosine transform is the core algorithm for the JPEG image compression standard. Here images and their subsets are compressed by discarding the higher order coefficients of the cosine transform, and decompressed using the inverse cosine transform, as in the example above. Special chips exist that implement these algorithms with high performance rates. The conserved information is the gridded "carpenter square" in the figure at left, set by 'p', i.e: the "compressed".

Mask

A C8

Part II: The obscure process of storing and recovering the "compressed" At this point, it is essential the reader understands how the conserved information is stored and recovered. It is maths again, array maths. Here we demonstrate "a" principle of the process. The "mise en oeuvre" may vary between different machine technology, but the principle is invariant as far as the result. The terminology is not standardized like chemical formulas but again the meaning is the function description.

 1 2 3  Assume we want to store (transmit too) the 1rst row and 1rst column of the array at left. We then " unwrap " the array into a vector  2 0 0    ... ()1 2 3 2 0 0 3 0 0 , then drop the 0's  3 0 0  ... ()1 2 3 2 3 , that is our "carpenter square" stored.

From the mass stoarage, the recovery is the inverse operation of the storage. We re-install the 0's dropped, and " pack " the vector into an array, thus recovering the original "carpenter square". 2 From the original matrix 'A' that contained 24 = 576 pieces of information we have dropped 2 ... 20 = 400 , i.e: a conservation ratio of ()576− 400 ÷ 576 = 31 % (QED) Conclusion : An image (JPEG, MPEG) is intentionally destroyed to a degree of acceptable subjective review. Such images CAN NOT be restored at the stage of the pellicule. Whatever can be imagined to restore would be artefacts only. This consideration is of the utmost importance in a Court Room when single frame of MPEG are projected on the screen trying to finger print some detail from minuscule destroyed portion of a whole destroyed as well.. JPEG is static, MPEG are dynamic. As such, sequences of MPEG can only prove some suite of actions. A single MPEG projected on the screen for eventually identifying an unknow yet evidence in small area of the screen is simply a technical idiosyncrasy ... the MPEG matrix from mass storage is the destroyed image matrix from the camera viewed scene. Also, consider spoiling by transmission "crackers" between the camera and the mass storage if not hard wired, also spoiling between the CD inserted in the video to screen converter as well as proof fabrication by zelous investigation ... all sorts of "destruction of the proof". Data statistically similar to random noise may not be compressible by any means. Noise from open-air transmission Noise inducted in hard-wired system Camera sensors fooled by laser beam Depending upon what part of the "good signal" is noise-polluted, will project on the screen as blured region or simply false ... etc. The "circumstantial" in MPEG CSI single shot on the screen is that at one time a true image may have passed through the camera lens and on the screen there is no more "true image". Who toke the details out of the image ? Fourier DFT... NO ! ______The defendant ... NO ! Somebody else ... likely probable The technology ... YES ! Quod Erat Demonstrandum of JPEG/MPEG destruction: Below, the Arc demo is exagerated as we take 96 pixels. In reality, JPEG/MPEG decomposes the captured image in multiple sub-images of smaller size thus enabling a visual fuzzy view, but not to the point to appreciate any detail unless savantly reconstucted from geometric pattern recognition, but that is not the case for unidentifiable shapes.

Example 2 ... destroy beautiful images L := Arc := 0 1 0 1 0 86 99 0 126.984 171.429 1 92 108 1 126.984 126.984

levels := 6 L6:= imquant() L, levels

At some point in the JPEG/MPEG process there is quantization. Quantization may dramatically destroy or transform the final image. In the 6 levels (L6) Lena if you think you see a nose piercing, then wrong !

L6 Just below, we re-introduce the reader to the DCT/IDCT. A geometric pattern is chosen to demonstrate the generality of the the perfect DCT/IDCT by itself.

i := 0 .. rows() Arc − 1  max() Arc   200    =   j:= i  min() Arc   0 

Adct:= dct2d() submatrix() Arc , 0 , 95 , 0 , 94  max() Adct   14091    =   Arec:= idct2d() Adct  min() Adct   −2791 

The scaled recovered Arc is scale() Arec , 0 , 200 it shows as the exact replicate to ± 12 d

scale() Arec , 0 , 200 scale() Adct −, 2791 , 14091

Now we take beautiful Arc and make a JPEG like picture to demonstrate the total destruction at 25%, much of the details lost @ 50% and an acceptable reconstruct @ 75% Define a "carpenter square factor" csf Define the % conservation area

csf := 2 2 rows() Arc − ()rows( Arc )− csf 13 Γ := 2 rows() Arc csf = 13

Arc_costr & icostr

Note: In the PDF document you see two rectangles with numbers 4 and 13 ... these, in the work sheet are the Mathcad sliders (also found as Microsoft tool) .

csf = 13 = Γ = 25 %

Live JPEG 2000

Two 2D wavelets are used in applications involving images, matrices, and other 2D data. The properties which make wavelets attractive for analysis of 1D functions hold for 2D functions as well. In particular, wavelets have been found to be very effective for image coding and data compression, since relatively few coefficients are needed to represent the image.

JPEG 2000 is a wavelet-based image compression standard. It was created by the Joint Photographic Experts Group committee with the intention of superceding their original discrete cosine transform-based JPEG standard. The standardized filename extension is .jp2 for ISO/IEC 15444-1 conforming files and .jpx for the extended part-2 specifications, published as ISO/IEC 15444-2, while the MIME type is image/jp2.

JPEG 2000 can operate at higher compression ratios without generating the characteristic 'blocky and blurry' artifacts of the original DCT-based JPEG standard. It also allows more sophisticated progressive downloads.

Part of JPEG 2000 has been published as an ISO standard, ISO/IEC 15444-1:2000. As of 2007, JPEG 2000 is not widely supported in web browsers, and hence is not generally used on the World Wide Web. Even though provisions have been made to integrate all kinds of external meta-data, there is currently no accepted way to embed Exif data.

Disadvantages / Legal issues

JPEG 2000 is not widely supported in present software due to the perceived danger of software patents on the mathematics of the compression method, this area of mathematics being heavily patented in general. JPEG 2000 is by itself not license-free, but the contributing companies and organizations agreed that licenses for its first part – the core coding system – can be obtained free of charge from all contributors.

Image Compression Using Wavelets

Wavwlet maths, Lena matrix

Original Image Now apply wavelet compression to an image. Use wavesort and shrink together to keep only the largest coefficients of the DWT.

Below at left we have conserved 10% of the coefficients ... at right with J := 4 only 1% of the coefficients are conserved . In the work sheet, J is the filter factor.

L

Wavelet maths %

Approximation with 10% of the coefficients Approximation with 1% of the coefficients The reproduction of the image is excellent With 1% of the coefficients, the quality of with only 10% of the coefficients retained. the image is drastically reduced, but the The fine detail in the hat is preserved, as are photograph is still recognizable. edges in the image.

scale() L1 scale() L2

Arc := 0 1 Below at center we have conserved 20% of the coefficients ... at right with J := 4 only 5% 0 126.984 171.429 of the coefficients are conserved . In the work 1 126.984 126.984 sheet, J is the filter factor.

Wavwlet maths, Lena matrix

Wavelet maths % Wavelet maths %

Original 20% of the coeff's 5% of the coeff's

Arc scale() Arc 1 scale() Arc 2

Since the amount of digital image data being stored and transmitted has increased so much in the past decade, compression algorithms and software are increasingly important. One method popular for of images is . Rather than encoding an image directly, the image is first transformed and the transform coefficients are then encoded. The data are compressed by the encoding step, but the sparsity of the transform coefficients determines the efficiency and accuracy of the encoding scheme.

The current commercial standard for lossy compression of digital images is the JPEG (Joint Photographics Experts Group) transform coding algorithm. JPEG is commonly available in many commercial software and imaging systems. Under the JPEG standard, blocks of 8x8 picture elements (pixels) are transformed using the discrete cosine transform (DCT), a widely used transform in signal and image processing. The JPEG standard then encodes the resulting DCT coefficients. For many images, however, the can be sparser than the DCT, and hence can lead to less distortion at higher compression ratios. The efficient edge representation of the wavelet transform can also lead to fewer artifacts in the compressed image.

The image below displays a magnetic resonance image (MRI) of a human brain, along with the magnitude of the wavelet transform coefficients. Darker pixels denote larger coefficients. The wavelet transform decomposes the original image into sub-images at different scales and locations. The large wavelet coefficients tend to correspond to important features in the image, such as edges. The figure below shows that the wavelet transform is very sparse compared with the original image: most of the wavelet coefficients are close to zero. The sparsity of the wavelet transform is the key for compression efficiency. A very sparse transform, which has fewer “important” coefficients to encode, can be efficiently compressed.

Brain := 0 1 2 3 4 0 1 2 3 4 0 0 0 0 0 0 1 1 2 2 2 2 2 1 4 5 1 2

γ = Ajust_Contrast

γ := 253

Brain

T Brain := γ − submatrix( Brain, 0 , cols() Brain − 1 , rows() Brain − 2 , 0)

Brain dwt := dwt() Brain , 3 , coiflet ()12

MRI Wavelet transform coefficients

Brain → 200 − Brain dwt

Wavelet compression is a form of data compression well suited for image compression (sometimes also video compression and audio compression). The goal is to store image data in as little space as possible in a file. Wavelet compression can be perfect, lossless data compression, or a certain loss of quality is accepted with lossy data compression.

Using a wavelet transform, the wavelet compression methods are adequate for representing transients, such as percussion sounds in audio, or high-frequency components in two-dimensional images, for example an image of stars on a night sky. This means that the transient elements of a data signal can be represented by a smaller amount of information than would be the case if some other transform, such as the more widespread discrete cosine transform, had been used. Wavelet compression is not good for all kinds of data: transient signal characteristics mean good wavelet compression - smooth, periodic signals are better compressed by other methods. Data statistically similar to random noise may not be compressible by any means.

[edit] Method First a wavelet transform is applied. This produces as many coefficients as there are pixels in the image (i.e.: there is no compression yet since it is only a transform). These coefficients can then be compressed more easily because the information is statistically concentrated in just a few coefficients. This principle is called transform coding. After that, the coefficients are quantized and the quantized values are entropy encoded and/or run length encoded.

Examples for wavelet compression: Still images JPEG 2000 MrSID ECW SPIHT Video Dirac Tarkin Pixlet Rududu [1] Snow

Digital fingerprints are created at a resolution of 500 pixels per inch and 256 gray scale levels (8 bits). As a result, a typical uncompressed fingerprint image requires roughly half a megabyte of storage. Wavelet methods were selected as part of an FBI standard for compression of finger print images. This section analyzes a fingerprint image using the approach in the FBI standard. This fingerprint image is a 512 x512 matrix. Handling a 512 x512 matrix is a lot of work. To speed computation, you can reduce this image to 256 x256 by performing a 2 x2 block average. This average can be done quickly using the wavelet transform with a Haar wavelet: The Proposed FBI Standard ... is based on a specific wavelet packet transform with 64 subbands, using a symmlet(8) wavelet.

Original ... Recovered

The original fingerprint reduced size Recovered as stored

F Rec This document, done & composed by Jean Giraud, june 02, 2004

ggggggggggggggggg Medical imaging: JPEG's 12-bit mode There are many medical imaging systems that create and process 12-bit JPEG images. The 12-bit JPEG format has been part of the JPEG specification for some time, but very few consumer programs (including web browsers) support this rarely used JPEG format .

JPEG quality (compression parameter) WinGrab

The degree of JPEG compression is determined by a quality parameter which can vary on a scale from 0 (strongest compression/lowest quality) to 100 (weakest compression/highest quality). The scale here is identical to that of the Independent JPEG Group.

Larger quality results in a larger image file and the output image becomes closer to the original file. A quality of 100 minimizes loss, but there is still information loss present because the present program does not support the so-called lossless JPEG process. JPEG is a lossy format, meaning that the output image is not identical to the input image.

Normally you want to use the lowest quality setting (smallest file) that decompresses the image into something visually indistinguishable from the original image. For this purpose the quality setting should be between 50 and 95, the optimal quality depends on the actual image. If you see defects in the image, try to increase the parameter by 5 or 10.

Quality values above 95 are not recommended for normal use; the compressed file size goes up for hardly any visible gain in output image quality. If you really want to avoid image loss, you should use a lossless format such as PNG (portable network graphics).

Quality values below 50 will produce very small files of low image quality. Settings around 5 to 10 might be useful in preparing an index of a large image library. Try a quality as low as 2 (or so) for some amusing Cubist effects. (Note: quality settings below about 25 generate so-called 2-byte quantization tables, which are considered optional in the JPEG standard and the file format becomes 'Extended sequential' instead of 'Baseline'. Some JPEG decoders may be unable to read such files.)

The standard (default) JPEG quality (see File->Options->JPG) suggested by the Independent JPEG Group is 70.

Standard JPEG format Standard JPEG format is Baseline JFIF 1.01 (JPEG File Interchange Format). (This format is readable by all JPEG decoders on planet Earth).

Full color images (24 bits/pixel) or images with 256 colors or 256 levels of grayscale (8 bits/pixel) can be saved as JPEG. Black/White (1 bit/pixel) or 16 color bitmaps (4 bits/pixel) can not be saved as JPEG.

Optimized JPEG format Optimizing (default) performs optimization of the so-called parameters. This usually makes the JPEG file a little smaller, but the program runs slower and needs more memory. Image quality and speed of decompression is unaffected by optimizing. Use optimizing if the size of the JPEG file is most important. Options for reading JPEG A JPEG image can be decompressed to different sizes. For example an image with original dimension 800x400 pixels can be decompressed to any of:

1/1 size (800x400 pixels) 1/2 size (400x200 pixels) 1/4 size (200x100 pixels) 1/8 size (100x50 pixels).

Decompressing to a smaller size is much faster than decompressing to full size.

If you want to view images (use program as an image-viewer) it is useful to let the program choose the decompressed size which best ’matches’ the size of the program window - this avoid the slow decompression to full size of large images and makes the program to a fast image-viewer.

If you want to do something with images (not only view them, but print or edit them in any way) it is highly recommended to start with the original image i.e. decompress it to full size. Thus you are using the highest quality version of the image as you starting point , and not some small low quality image.

JPEG images are normally decompressed to full color images (24 bits/pixel) but can optionally be decompressed to palette images or to grayscale images (256 colors or 256 gray levels, 8 bits/pixel) with a significant faster decompression. Decompressing to 256 colors is better than decompressing to full color followed by a color reduction. Decompressing to 256 gray levels could be useful when you have a printer with no color capabilities (laser printer).

Progressive JPEG format In a 'progressive JPEG file' the data is stored in multiple scans of increasing quality. If the file is being transmitted over a slow communications line (for example Internet) the decoder can use the first scan to display a low-quality image very quickly, and can then improve the display with each subsequent scan. The final image is exactly equivalent to a standard JPEG file of the same quality setting, and the total file size is about the same - often a little smaller.

Progressive JPEG was not widely implemented until 1996 (some older programs like PaintShopPro ver 3) does not support it) but are now in common use on the Internet. Netscape Navigator 4.04 utilizes the progressive display but Microsoft Internet Explorer 5 does not. If you publish a large jpg image (say 50 - 100 kB or more) on Internet, then a progressive JPEG should be considered.