School of Electrical and Computer Engineering Inkjet Forensics
Maria V. Ortiz Aravind K. Mikkilineni http://shay.ecn.purdue.edu/~prints
1 School of Electrical and Computer Engineering About the Presenters
• Maria V. Ortiz – received her B.S. from the Universidad Javeriana, Bogota-Colombia in 2005. She is a Graduate Student at Purdue University where she holds a Research Assistantship on the department of Electrical and Computer. Schlumberger awarded her a scholarship as a recognition of her academic effort in 2004. Her thesis, based in the recognition of the Malaria, was nominated to “Best Engineering Thesis 2005”
• Aravind K. Mikkilineni - received his B.S. in Electrical and Computer Engineering from Ohio State University in Columbus, Ohio in 2002. In 2004 he received his M.S. in Electrical and Computer Engineering from Purdue University while working as a research assistant in the area of printer security and forensics. He is currently working toward a Ph.D. at Purdue University while continuing work on printer and device forensics.
September 27, 2006 ENFSI-EDEWG 2
2 School of Electrical and Computer Engineering Electronic Imaging Systems Laboratory • Prof. Jan P. Allebach
• Investigate imaging systems (printers, scanners, digital cameras and displays) • Characterize imaging systems – Perform measurements and analysis on imaging systems – Identify problematic sources in black box models • Improve imaging systems – Test solutions on systems – Test solutions on human participants (psychophysics experiments) – Improve solutions model to integrate the human perception – Implement solution model
September 27, 2006 ENFSI-EDEWG 3
3 School of Electrical and Computer Engineering VIPER Lab
• Video and Image Processing • Prof. Edward J. Delp
• Video Coding • Device Forensics • Watermarking • Document Security • Sensor Networks
September 27, 2006 ENFSI-EDEWG 4
4 School of Electrical and Computer Engineering Agenda
• Introduction to digital printing
• Basic inkjet printing process • Digital Imaging • Inkjet details
• Short Lab – (Identifying print process parameters)
• Print modes (print driver settings) • Examination of Print Samples
[#] indicates a reference which are listed at the end of the presentation
September 27, 2006 ENFSI-EDEWG 5
5 School of Electrical and Computer Engineering Basic Inkjet Printing Process
• Paper path • Carriage • Cartridge • Print head • Inkjet printing technologies
September 27, 2006 ENFSI-EDEWG 6
The paper path along with the carriage, the cartridge, and the printhead are involved in every inkjet printing process and their interaction depends on the printing technology of the printer.
6 School of Electrical and Computer Engineering Overview of the Inkjet Process
September 27, 2006 ENFSI-EDEWG 7
7 School of Electrical and Computer Engineering Paper path
Paper is advanced through the printer A carriage transports the pen back and by a series of rollers driven by a forth across the page. The pen fires ink [3] stepper motor. onto the surface of the page.
September 27, 2006 ENFSI-EDEWG 8
Red arrows show the printing direction Blue arrow shows the process direction
8 School of Electrical and Computer Engineering Carriage
September 27, 2006 ENFSI-EDEWG 9
These pictures were taken from the printer model we used in the demo session The region highlighted in red is an optical encoder strip which is used by the printer to position the carriage for proper dot placement. The image on the right shows the carriage which carries the printhead and the cartridge/cartridges. This printer uses two cartridges, one for color and the other for black.
9 School of Electrical and Computer Engineering Cartridge
[2][9][21]
September 27, 2006 ENFSI-EDEWG 10
Different types of cartridges. The cartridges on the right do not carry the printhead. Instead, in this kind of printer, the printhead is fixed to the carriage in the printer. The middle cartridges have the printhead attached to them and as a consequence the printhead is replaced with the cartridge. On the left, we can see the electrical contacts between the cartridge and the printer.
10 School of Electrical and Computer Engineering Printhead/Nozzles
1/600 in
1/300 in
1/600 in
September 27, 2006 ENFSI-EDEWG 11
The printhead is composed of nozzles which eject drops of ink out of the cartridge and onto the paper. This diagram illustrates the nozzle placement for a printer with a resolution of 600dpi . Since the separation between them is very small, the nozzles are arranged in multiple offset columns (in this case two) because of mechanical limitations and to help avoid coalescence where two adjacent drops will merge into one drop.
11 School of Electrical and Computer Engineering Inkjet Printing Technologies
[1]
September 27, 2006 ENFSI-EDEWG 12
Inkjet printing is divided into continuous and drop-on-demand technologies. In the continuous inkjet technology the drops are formed continuously but only some of them are expelled, on the other hand, in the drop-on-demand inkjet technology the drops are generated as needed.
12 School of Electrical and Computer Engineering Thermal Inkjet
Side-shooter Roof-shooter
[1]
September 27, 2006 ENFSI-EDEWG 13
The thermal inkjet technology is divided in two categories according to the position of the heater element, side shooter and roof shooter. As the temperature increases, the heater and the ink get hotter, until a bubble is formed and by the pressure in the ink reservoir, a drop is fired out the cartridge.
13 School of Electrical and Computer Engineering Piezoelectric Inkjet Bend mode
Classification depends on the piezoceramic deformation mode. Four main types:
• Squeeze-mode can be designed with a thin tube of piezoceramic surrounding a glass nozzle or with a Push mode piezoceramic tube cast in plastic that encloses the ink channel. • Bend-mode design, the piezoceramic plates are bonded to the diaphragm forming an array of bilaminar electromechanical transducers used to eject the ink droplets. • Push-mode design, as the piezoceramic rods expand, Shear mode they push against ink to eject the droplets. • Shear-mode the electric field is designed to be perpendicular to the polarization of the piezodriver. The shear action deforms the piezoplates against ink to eject the droplets.
[1] September 27, 2006 ENFSI-EDEWG 14
On the piezoelectric printers, the deformation of a piezoceramic material reduces the space inside the ink chamber and a drop is expelled. There are four categories depending on the piezoceramic element used.
14 School of Electrical and Computer Engineering Digital Imaging
Image formation and associated artifacts
September 27, 2006 ENFSI-EDEWG 15
Understanding digital imaging artifacts and being able to distinguish them from printer artifacts is essential.
15 School of Electrical and Computer Engineering Digital versus Printer Artifacts
Stepping: the imperfection of drawing of long lines sloped at a very small angle.
[13] [19]
September 27, 2006 ENFSI-EDEWG 16
Stepping, the jaggedness seen on edges of straight or curved lines, is a digital artifact that could be mistaken for a printer artifact.
16 School of Electrical and Computer Engineering Halftoning
September 27, 2006 ENFSI-EDEWG 17
Understanding textures created by the process of halftoning are also important to understand. Small dots visible in light halftoned regions should not be mistaken for stray printer dots.
17 School of Electrical and Computer Engineering Inkjet Details
• Performance vs. image quality • Ink drop formation and ejection • Dot structure • Swath alignment and Passes • Multi-pass, single-pass, and print masks • Page edge artifacts • Paper interaction • Ink types
September 27, 2006 ENFSI-EDEWG 18
18 School of Electrical and Computer Engineering Performance Versus Image Quality Hardware Logic Printhead Velocity Pen size and Firing Frequency
Print Mechanism
Output Resolution
Ink/Media Ink Ability to Mask Defects [21] Capacity Ink Flux
September 27, 2006 ENFSI-EDEWG 19
Every process involved in creating an inkjet print is important and cause an inherent tradeoff between printing speed/performance and print quality.
19 School of Electrical and Computer Engineering Drop Ejection
[14]
September 27, 2006 ENFSI-EDEWG 20
The drops are ejected out of the cartridge though the nozzles. As the drop falls from the cartridge to the media, it splits into the main body or primary drop and the secondary drop. The secondary drop should fall on top of the primary one but some aerodynamic effects prevent such event to happen. Notice that some of the ink is soaked up again by the empty space that was left in the ink reservoir.
20 School of Electrical and Computer Engineering Dot Structure
Satellite
Tail
[3][4]
September 27, 2006 ENFSI-EDEWG 21
When the secondary drop does not fall on the same spot as the primary drop, it becomes either a satellite of a tail. Satellite : secondary dot is not attached to the main dot Tail : secondary dot is attached to the primary dot Using the position of the satellites or tails you can find out the printing direction Left to right: satellite/tail on the right side of the main dot Right to left: satellite/tail on the left side of the main dot leading the printing direction.
21 School of Electrical and Computer Engineering Effect of Carriage Speed on Dot Structure
15 ips left to right 45 ips left to right 45 ips right to left
[3]
September 27, 2006 ENFSI-EDEWG 22
A slow print speed causes satellite drops to fall on top of or near the main drop as seen in the left image. As the print speed increases, the satellite drop falls further from the main drop as seen in the right two images.
22 School of Electrical and Computer Engineering Dot Placement Error Horizontal dot displacements Horizontal dot displacements for even raster for odd raster
Vertical dot displacements for even raster Vertical dot displacements for odd raster
[4]
September 27, 2006 ENFSI-EDEWG 23
Due to the position of the nozzles on the printhead (i.e. the space between the two columns) there is a misplacement of the dots in the horizontal position while in the vertical position there is not such error.
23 School of Electrical and Computer Engineering Swaths and Passes
• Swath height: length of printhead. Fixed size. • Alignment: manual or automatic. • Misalignment due to velocity and printing mode. [3][9]
September 27, 2006 ENFSI-EDEWG 24
Swath height: length of the printhead. In the diagram the size of the swath is 8, the first swath is printed from left to right and in the picture of the left you can see the spray/satellites on the edge, the second one is printed from right to left. The apparent misalignment from swath to swath is due to the high velocity of the carriage.
24 School of Electrical and Computer Engineering Single Pass Printing
Dark Swath Light Swath Horizontal Overlap
Horizontal Gap [21] September 27, 2006 ENFSI-EDEWG 25
In the single pass mode the printer fires the nozzles one time per swath. The horizontal overlap/gaps are caused by an error in the advance of the paper and/or misdirected nozzles which are not firing perpendicularly and therefore such nozzles cover either more or less than the swath height. The dark and light swaths in a bidirectional printer are caused by the difference in the aerodynamics of each swath (i.e in one swath the carriage moves from L to R and on the next one moves from R to L). This aerodynamic difference makes the satellites fall closer/further in one direction than in the other.
25 School of Electrical and Computer Engineering Multipass Printing • Print a given area in multiple passes – Only some fraction of the dots in a swath are printed in each pass
• Minimize swath-to-swath errors – Gaps/Overlaps –Hue shifts
• Requires use of a print mask
September 27, 2006 ENFSI-EDEWG 26
Goals of the multipass mode
26 School of Electrical and Computer Engineering Print Masks
Vertical position 1 0 1 0 Pen Sweep of pen for the 1st pass 0 1 0 1 Direction 1 0 1 0 0 1 0 1 0 1 0 1 1 0 1 0 Media 1 0 1 0 0 1 0 1 Advance 0 1 0 1 1 0 1 0 Direction 1 0 1 0 0 1 0 1 0 1 0 1 1 0 1 0 0 1 0 1 Vertical position 1 0 1 0 of pen for the 2nd pass
[3][4]
September 27, 2006 ENFSI-EDEWG 27
Example of two-pass printing mode. The swath is divided by the number of passes. In the first pass the printhead prints on the positions filled with ones in the print mask then the paper is advanced only half of the swath and the printhead uses the other half of the print mask to print. At the end of the second swath area to be printed in the rectangle will be finished.
27 School of Electrical and Computer Engineering Multipass Printing
[21] September 27, 2006 ENFSI-EDEWG 28
Another example of the multipass mode
28 School of Electrical and Computer Engineering 4-pass Printing
1st Pass 2nd Pass 3rd Pass 4th Pass
September 27, 2006 ENFSI-EDEWG 29
The swath is divided by four (number of passes). In each pass only some spaces are filled and the paper is advanced one quarter of a swath height. At the end the top section will be finished.
29 School of Electrical and Computer Engineering Page Edge Defects
• Dot structure different during carriage acceleration and deceleration
Accelerate Constant velocity Decelerate
September 27, 2006 ENFSI-EDEWG 30
The dot structure changes on the edges of the page because of the acceleration and deceleration of the carriage. The satellites/tails may look different from the ones present when the velocity is constant.
30 School of Electrical and Computer Engineering Paper-Ink Interaction
[1][16]
September 27, 2006 ENFSI-EDEWG 31
31 School of Electrical and Computer Engineering Paper-Ink Interaction
Paper samples 1 to 4 are inkjet coated papers, paper 5 is a polymeric laminate and paper 6 and 7 are coated copy papers. Paper 3 is the best coated paper and therefore its interaction with the ink is very similar to the laminate paper. [11]
September 27, 2006 ENFSI-EDEWG 32
32 School of Electrical and Computer Engineering Paper-Ink Interaction
A noticeable defect with inkjet printing is wicking or feathering. This defect is caused when ink flows along or within cellulose fibers.
Cockle effect: it happens when the paper gets wet from the ink
Difference in dot spreading between coated paper (left) and uncoated paper (right) [7][10][12][6]
September 27, 2006 ENFSI-EDEWG 33
33 School of Electrical and Computer Engineering Ink Types
[1]
September 27, 2006 ENFSI-EDEWG 34
34 School of Electrical and Computer Engineering Ink Types
Dye-based ink
Pigment-based ink 1
[1][15] Pigment-based ink 2
September 27, 2006 ENFSI-EDEWG 35
35 School of Electrical and Computer Engineering Demo
Print the provided test page using each of N modes. Determine as much as you can about the print process parameters chosen by the driver for each of these modes.
You should try to answer the following questions for each mode without physically examining the printer:
Is the test page printed in single or bi-directional mode? Can you approximately guess the print speed by examining the page (fast or slow)? Was a multi-pass mode used? Is the page printed in only K, CMY, or a mix of both (CMYK)? What is the swath height? Are there different swath heights for K and CMY? How many columns of nozzles are there for K? Can you identify defects on the page due to the choice of these print process parameters?
These questions are not exhaustive of the type of information that might be gathered from a printout, and it is possible that not all of the above questions can be answered for the test print in a given mode.
September 27, 2006 ENFSI-EDEWG 36
36 School of Electrical and Computer Engineering Print Modes
• Print speed • Number of passes • Number of direction • Drop volume • Resolution
September 27, 2006 ENFSI-EDEWG 37
37 School of Electrical and Computer Engineering Example Draft Mode
• Fast print (carriage) speed – Satellites and tails • Single pass/Bidirectional printing – Visible swath boundaries –Hue shifts • Lower resolution – Larger drop size
September 27, 2006 ENFSI-EDEWG 38
38 School of Electrical and Computer Engineering Example High Quality Mode
• Slow print (carriage) speed – Minimize satellites and tails • Multi-pass printing/Single direction – Reduce visibility of swath boundaries • High resolution – Small drop volume
September 27, 2006 ENFSI-EDEWG 39
39 School of Electrical and Computer Engineering
Print Samples
September 27, 2006 ENFSI-EDEWG 40
Print samples (provided by Jan de Koeijer)
40 School of Electrical and Computer Engineering Print Sample 1
• Bi-directional (Aerodynamics) – More dot-gain in one direction • Pen defects • If thermal IJ – heat buildup – Difficult to tell with vignetting
September 27, 2006 ENFSI-EDEWG 41
Characteristics: bidirectional printer, probably single pass mode. Possible explanations: · Assumption: swath size is one text line. There is more dot gain in one direction (even text lines) than in the other (odd text lines) because of the aerodynamics. The satellites sometimes fall very close or on top of the main drop, and sometimes far away. · Assumption: swath size is two text lines. Half of the pen is damaged and therefore the nozzles are not printing properly. · Assumption: temperature effects - the edge of the page from where the carriage starts the text line is lighter than the end of it because the temperature as well as the volume of ink is lower in that part. Difficult to tell because of vignetting
41 School of Electrical and Computer Engineering Print Sample 2
• Swath discontinuity • Aerodynamics – combing magnitude difference • 2 columns of nozzles
September 27, 2006 ENFSI-EDEWG 42
Characteristics: bidirectional printer. First segment printed from right to left because of the position of the combing, second segment printed on the opposite direction. Possible explanations: · Assumption: swath size is the length of the second segment or longer. Draft mode. Because of the distance between the indentations it seems the pen has two columns of nozzles. There is more combing on one segment than in the other because of the aerodynamics of each direction.
42 School of Electrical and Computer Engineering Print Sample 3
• Split in character – Smart passing – Not done in draft modes • Ascenders/Descenders – Nozzles used a lot fire well – Nozzles not used a lot • Run, more spray, offset dots • Combing – column spacing –2 column
September 27, 2006 ENFSI-EDEWG 43
Characteristics: splits in characters of the ascenders of the first and third line Possible explanations: · Assumption: split occurs on the swath boundary. The misalignment between swaths and some misdirected nozzles could cause the splits. · Assumption: nozzles used a lot fire well The nozzles of the ascenders are not used to fire (there are only two ascenders ‘b’ and ‘d’) as abundantly as the ones of the descenders, consequently these nozzles produce more spray and offset dots. · Smart passing: on some printers the driver can decide to fit as much text as possible in one swath, in order to avoid the splits in misalignment in the characters, and to put the swath boundary in the blanks between the text lines (not used in draft mode)
43 School of Electrical and Computer Engineering Print Sample 4
• Aerodynamic effects • Nozzle firing history
September 27, 2006 ENFSI-EDEWG 44
Characteristics: this is not common. Possible explanations: · Assumption: aerodynamic effects
44 School of Electrical and Computer Engineering Print Sample 5
• Draft style • High carriage speed • Lots of spray
September 27, 2006 ENFSI-EDEWG 45
Characteristics: notice the presence of a lot of spray on the left edge of the characters. The printer direction: right to left. Possible explanations: · Assumption: draft mode. The high velocity of the carriage in draft mode cause spray
45 School of Electrical and Computer Engineering Print Sample 6
•1st line bold? •2nd line gray (not black)?
September 27, 2006 ENFSI-EDEWG 46
Characteristics: see slide 41 Possible explanations: · Assumption: first line bold · Assumption: second line could be gray instead of black on the document
46 School of Electrical and Computer Engineering Print Sample 7
• Combing phase change could indicate swath boundary • Dead nozzles or misdirected nozzles
September 27, 2006 ENFSI-EDEWG 47
Characteristics: change of phase in the combing and white horizontal lines inside the character. Possible explanations: · Assumption: swath boundary The swath boundary could originate the change of phase on the combing (between the red lines) of the character. · Assumption: misdirected or dead nozzles The white lines within the character are due to some damaged/misdirected or dead nozzles. The misdirected nozzles do not fire perpendicularly so they could lead to have white spaces.
47 School of Electrical and Computer Engineering Print Sample 8
• Black and Color • Black left-to-right • Color right-to-left
September 27, 2006 ENFSI-EDEWG 48
Characteristics: black was printed from left to right and color from right to left. Possible explanations: · Assumptions:
48 School of Electrical and Computer Engineering Print Sample 9
• 2-pass – tails on both sides • Possible bad pen – Bad spray and/or tails
September 27, 2006 ENFSI-EDEWG 49
Characteristics: two pass printing mode (there are tails on both sides of the characters). Possible explanations: · Assumptions: bad pen.
49 School of Electrical and Computer Engineering Print Sample 10
• Smear – Built up fibers, pet hair, etc.
September 27, 2006 ENFSI-EDEWG 50
Characteristics: the tails have different angles. Possible explanations: · Assumptions: fibers, pet hair and any kind of dirt can be attached to the carriage.
50 School of Electrical and Computer Engineering Print Sample 11
• Draft mode • Piezo-electric – Low firing frequency – Larger drop size to cover same area • Lots of spray
September 27, 2006 ENFSI-EDEWG 51
Characteristics: piezoelectric printer. Spray Possible explanations: · Assumptions: draft mode Due to the low firing frequency of the piezoelectric printers, the dots need to be larger to cover the same area in a draft mode where a high carriage speed is used.
51 School of Electrical and Computer Engineering Print Sample 12
• Draft mode • Single pass • Bi-directional • Spray very far from main drops •1st, 5th and last line look like different type of text – Print head temperature
September 27, 2006 ENFSI-EDEWG 52
52 School of Electrical and Computer Engineering Conclusions
• Inkjet printing is complex
• Artifacts from a specific printer will differ depending on the settings of the print driver, content printed, paper type, ink type, etc…
September 27, 2006 ENFSI-EDEWG 53
53 School of Electrical and Computer Engineering References
[1] Hue P. Le, Progress and Trends in Ink-jet Printing Technology , Journal of Imaging Science and Technology, Volume 42, Number 1, January/February 1998, pp. 49–62. [2] Jeff Tyson, How inkjet printers work, www.howstuffworks.com [3] Edgar Bernal, Improved pen alignment for bidirectional printers. [4] Osman Aslan, Gazi Ali et al, Print quality issues related to digital printing and forensic applications. [5] Eric Hanson, How an ink jet printer works, Hewlett Packard Laboratories. [6] Computer Friends Inc, Travel to the center of an inkjet cartridge. [7] Dave Brooks et al, Improvement of Ink Jet Printer Performance by Modifying Office Papers, Ink-Jet Components Division, Hewlett-Packard, San Diego, California. [8] Kenji Suzuki et al, Dynamics of Droplet Forming in Ink Jet Printer, Recent Progress in Ink Jet Technologies II, 1999. [9] Rob Beeson, Thermal Inkjet: Meeting the Applications Challenge, Hewlett Packard Company, Corvallis, Oregon. [10] Cheryl Katen, The Top 10 Breakthroughs in Thermal Ink Jet Technology, Hewlett-Packard Company, San Diego, CA. [11] Anne Mähönen et al, The Splashing of Ink Drops in CIJ Printing, VTT Information Technology Espoo, Finland.
September 27, 2006 ENFSI-EDEWG 54
54 School of Electrical and Computer Engineering References
[12] Makoto Torigoye, High-quality Imaging Technologies for Color Bubble Jet Printers, BJ Printing System Development Center, Canon Inc., Japan, 1999. [13] Ludwik Buczynski, Special Print Quality Problems of Ink Jet Printers, Warsaw University of Technology, Mechatronic Department, Warsaw, Poland, 1999. [14] Hongming Dong et al, An experimental study of drop-on-demand drop formation, Physics Of Fluids 18, 2006 [15] Joseph E. Johnson and James A. Belmont, Novel Black Pigment For Ink Jet Ink ApplicationsCabot Corporation, Billerica, Massachusetts. [16] Akira Asai et al, Impact of an Ink Drop on Paper Canon, Inc., Kanagawa, Japan. [17] Leonard Carreira et al, The Effect of Drying Rate on Inter-Color Bleed Xerox Corporation, Webster, New York. [18] Aidan Lavery and John Provost, Color-Media Interactions in Ink Jet Printing, Zeneca Specialties, Manchester, UK [19] www.alpenglowimaging.com/images/pixels.jpg [20] He-Jo Lee and Jan Allebach, Inkjet printer model based halftoning, IEEE transactions on image processing, vol 14, no. 5, May 2005. [21] Morgan Shramm, HP Lab University, June, Paris, 2006.
September 27, 2006 ENFSI-EDEWG 55
55 School of Electrical and Computer Engineering
Questions/Comments?
September 27, 2006 ENFSI-EDEWG 56
56