Watermarking Spot Colors in Packaging

Home , Chrominance, Colorfulness

This paper was published in the proceedings of the 2015 IS&T/SPIE Electronic Imaging conference: Media Watermarking, Security, and Forensics, San Francisco, CA February 8, 2015, volume 9409 and is made available as an electronic reprint with permission of SPIE. Single print or electronic copies for personal use only are allowed. Systematic or multiple reproduction, or distribution to multiple locations through an electronic list server or other electronic means, or duplication of any material in this paper for a fee or for commercial purposes is prohibited. By choosing to view or print this document, you agree to all the provisions of the copyright law protecting it.

Alastair Reed, Tomáš Filler, Kristyn Falkenstern, Yang Bai Digimarc Corporation, 9405 SW Gemini Drive, Beaverton, OR 97008, USA

ABSTRACT In January 2014, Digimarc announced Digimarc Barcode for the packaging industry to improve the check-out efficiency and customer experience for retailers. Digimarc Barcode is a machine readable code that carries the same information as a traditional Universal Product Code (UPC) and is introduced by adding a robust digital watermark to the package design. It is imperceptible to the human eye but can be read by a modern barcode scanner at the Point of Sale (POS) station. Compared to a traditional linear barcode, Digimarc Barcode covers the whole package with minimal impact on the graphic design. This significantly improves the Items per Minute (IPM) metric, which retailers use to track the checkout efficiency since it closely relates to their profitability. Increasing IPM by a few percent could lead to potential savings of millions of dollars for retailers, giving them a strong incentive to add the Digimarc Barcode to their packages. Testing performed by Digimarc showed increases in IPM of at least 33% using the Digimarc Barcode, compared to using a traditional barcode. A method of watermarking print ready image data used in the commercial packaging industry is described. A significant proportion of packages are printed using spot colors, therefore spot colors needs to be supported by an embedder for Digimarc Barcode. Digimarc Barcode supports the PANTONE spot color system, which is commonly used in the packaging industry. The Digimarc Barcode embedder allows a user to insert the UPC code in an image while minimizing perceptibility to the Human Visual System (HVS). The Digimarc Barcode is inserted in the printing ink domain, using an Adobe Photoshop plug-in as the last step before printing. Since Photoshop is an industry standard widely used by pre-press shops in the packaging industry, a Digimarc Barcode can be easily inserted and proofed.

1. INTRODUCTION In January 2014, Digimarc announced Digimarc Barcode for the packaging industry to improve the check-out efficiency and customer experience for retailers.4 Digimarc Barcode is a machine readable code that carries the same information as a traditional UPC and is introduced by adding a robust digital watermark to the package design. It is imperceptible to the human eye but can be read by a modern barcode scanner at the POS station. Compared to a traditional linear barcode, Digimarc Barcode covers the whole package with minimal impact on the graphic design, and thus eliminates the need to search for the barcode at the checkout. This significantly improves the IPM metric, which retailers use to track the checkout efficiency since it closely relates to their profitability. Increasing IPM by a few percent could lead to potential savings of millions of dollars for retailers, giving them a strong incentive to add the Digimarc Barcode to their packages. Quantitative model of expected savings is available in Ref. 4. Testing performed by Digimarc showed increases in IPM of at least 33% using the Digimarc Barcode, compared to using a traditional barcode.8 Automation and workflow considerations for embedding Digimarc Barcodes at large scale has been presented in Ref. 16. Although digital watermarking is a well-established field with a long list of successful applications5 spanning various media such as audio, video and print, watermarking consumer package goods on a large scale brings new challenges not encountered before. In this paper, we address one such challenge caused by the use of special inks and a barcode imaging system, a case often seen in packaging and retail. Packages are printed with two common ink systems:

1. Process colors – Cyan, Magenta, Yellow and Black (CMYK) inks are used to simulate a wide range of colors, by mixing the ink on a substrate and printing half tone dots. This ink system is used in most consumer printers. E-mail: {Alastair.Reed, Tomas.Filler, Kristyn.Falkenstern, Yang.Bai}@digimarc.com; http://www.digimarc.com 2. Spot colors are custom pre-mixed inks designed to achieve a certain color when printed on a speciﬁed substrate.

The motivation to watermark spot colors is that a significant proportion of packages are printed using spot colors or contain some spot color regions. Packages are often printed with spot colors to reduce cost and for color accuracy and consistency. Other reasons for using spot colors in packaging are to obtain colors outside of the process color gamut, or to create special effects such as fluorescent∗, metallic or optically variable inks.15 Industry standard library of spot inks, published by PANTONE, contains several hundreds of inks. In packaging, Digimarc Barcodes are inserted in the chrominance (as opposed to luminance) domain to obtain the best robustness per unit visibility10 at a spatial frequency corresponding to 75 DPI (Dots Per Inch). The choice of spatial resolution allows Digimarc Barcode to be printed by typical offset and flexographic presses on a range of substrates, and read by modern imaging-based barcode scanners. Modern barcode scanners are monochrome imaging devices typically with red LED illumination. The red LED is a narrow-band light source with a wavelength of 660 nm, which implies that the scanner can only see image content on packages printed with inks that have low reflectivity at this wavelength, such as Cyan or Black. In this paper, we describe a set of algorithms along with working examples of embedding Digimarc Barcode in a package design. In general, the watermark embedding method minimize the visual impact of the added signal while achieving specified signal strength as seen by the POS scanner. This problem is solved for cases involving a mix of process and/or spot colors while considering practical limitations coming from actual presses. The watermark embed method described in Ref. 10 was enhanced to support multiple spot colors and detection with a barcode scanner. The method minimizes the impact on the printer workflow by maintaining colors used in the original design as much as possible. The Digimarc Barcode is inserted in the printing ink domain, using an Adobe Photoshop plug-in as the last step before printing. Since Photoshop is an industry standard widely used by pre-press shops in the packaging industry, a Digimarc Barcode can be easily inserted either internally by the Professional Services Group at Digimarc or externally by a pre-press partner. Standard proofing methods using Photoshop, internal or third party tools can be used to preview the results for visibility and robustness. Due to multidisciplinary nature of this paper, we first describe necessary print and color background in Section 2. Section 3 discusses embed constraints posed by POS scanner and possibly other imaging devices. Watermark embed in process inks and associated challenges are described in Section 4. Section 5 generalizes the embed strategies to handle both process and spot colors. Finally, a brief summary can be found in Section 6.

2. PRINT AND COLOR BACKGROUND This section includes preliminary material required to understand different aspects of color science and printing. Readers familiar with these areas are welcome to skip this section and return back when necessary. Key terms used in the paper are printed in bold. 2.1 Color and Human Visual System As mentioned, Digimarc Barcodes are inserted in the chrominance domain rather than luminance to reduce the visibility of the mark. In this section we briefly describe a few of the fundamental concepts which are used throughout the paper which are related to: color, color spaces and the visibility of color differences. 2.1.1 Color Perception The color of an object is the result of the interaction between a light source, an object and a detector (often the human visual system). Light is radiation which can be seen, in the wavelength range of about 380 to 780 nm. Spectral reflectance is used to describe how an object interacts with light. The spectral reflectance curves describe the fraction of light reflected at each wavelength from the object. When the reflected light is detected and interpreted through the visual system it results in an object having a particular color. The most common way to capture the spectral data with a device is by using a spectrophotometer. ∗Fluorescent inks are used to create eye-catching colors such as Tide orange (www.dayglo.com, en.wikipedia.org/ wiki/Tide\_(brand)). Red LED Paper Cyan Magenta Yellow Black 1931 CIE standard observer color matching functions

100% x¯(λ) y¯(λ) z¯(λ) 1.5 75% 1 50%

Reﬂectance 25% 0.5

0% 0 400 450 500 550 600 660 700 400 450 500 550 600 650 700 Wavelength λ nm Wavelength λ nm Figure 1. (Left) Spectral reﬂectance of PANTONE process inks as measured using X-Rite i1Pro spectrophotometer. Graph also shows spectrum emitted by red LED centered at 660nm. (Right) 1931 CIE 2° standard observer matching functions used for converting spectral reﬂectance to CIE XYZ color space.

2.1.2 Describing Color Often color is described by artists in terms of mixing paint. An artist often starts with white paper, which reflects most of the light. Different colored pigments are applied on top of the paper, which reduce the amount of light reflected back. Current trends for printing describe subtractive four color mixing using CMYK. Yellow, for instance, reflects most of the light, it absorbs only the lower wavelengths. Figure 1 shows the spectral reflectances of the PANTONE CMYK process inks. 2.1.3 CIE Color Spaces In 1931, the CIE (Commission Internationale de l’Eclairage) developed a way to link between wavelengths in the visible spectrum and colors which are perceived by the human visual system. The models which the CIE developed made it possible to transform color information between physical responses to reflectance in color inks, illuminated displays, and capture devices such as digital cameras into a perceptually (nearly) uniform color space. The CIE XYZ color space was derived by multiplying the color matching functions† with the spectral power of the illuminant and the reflectance of an object, which results in a set of XYZ tristimulus values for a given sample. Within the CIE model, CIE Y describes the luminance or perceived brightness. While the CIE X and CIE Z plane contain the chromaticities, which describes the color regardless of luminance. Chromaticity can be described by two parameters, hue and colorfulness. Hue or hue angle, describes the perceived color name, such as: red, green, yellow and blue. Colorfulness is the attribute which describes a color as having more or less of its hue. A color with 0 colorfulness would be neutral. The CIE took the CIE XYZ space to propose a pseudo-uniform color space, where calculated differences are proportional to perceptual differences between two color stimuli, formally referred to as the CIE 1976 L∗ a∗ b∗ (CIELAB) color space. The L∗ coordinate represents the perceived lightness, an L∗ value of 0 indicates black and a value of 100 is white. The CIE a∗ coordinate position goes between “redness” (positive) and “greenness” (negative), while the CIE b∗ goes between “yellowness” (positive) and “blueness” (negative). 2.1.4 Color Difference

To describe how perceptually similar two colors are, the CIE developed a color diﬀerence model, CIE ΔE76. The ﬁrst model developed was simply the Euclidean distance in CIELAB between two color samples. Since then, other more complex models have been developed to address some of the non-uniformity within the CIELAB color-space, most notably the sensitivity to neutral or near neutral colors. 101 75 DPI

0 10 75/2 DPI

Contrast 1 10− 25 DPI

Luminance Red-green Blue-yellow 10 2 − 10 75/2 75 150 Spatial resolution (DPI) Figure 2. Contrast sensitivity function of human eye. Plot shows inverse of magnitude required for luminance, red-green, and blue-yellow sine wave to become just noticeable as a function of frequency. Crosses show experimental data.6, 11

2.1.5 Contrast Sensitivity Function The CIELAB color diﬀerence metric is appropriate for measuring the color diﬀerence of a large uniform color region, however, the model does not consider the spatial-color sensitivity of the human eye. The luminance and chrominance CSF (Contrast Sensitivity Function) of the human visual system has been measured for various retinal illumination levels. The luminance CSF variation was measured by van Nes and Bouman (1967)11 and the chrominance CSF variation by van der Horst and Bouman (1969)6 and the curves are plotted in Figure 2 for a single typical illumination level. Typical watermark contains most signal energy over the spatial resolutions shown by the gray box. If the luminance and chrominance contrast sensitivity functions are integrated over this gray box region, the resultant energy ratios calculate the uniform perceptual scaling for CIELAB L∗, a∗ and b∗. 22 Thus the watermark perceptual error ∆EWM is calculated as

1 2 2 2 2 ∆EWM = ∆L + (∆a/8) + (∆b/16) , (1) where ∆L is the luminance variation and ∆a and ∆b the two chrominance variations introduced by the watermark signal.

2.2 Printing Processes A variety of technologies are used to print commercial packages, the most common of which may include: offset lithography, flexographic, gravure and digital printing. For a brief outline of each technology see Ref. 12 and for a more detailed explanation see Ref. 9. 2.2.1 Print Technologies In addition to the technology, various materials and techniques are used in the printing process which need to be considered for watermarking a spot color, these materials include: substrate, process colors, overprinting, spot colors, spot tint (screening) and process equivalent tints. In printing, substrate refers to the base material which a design is printed onto. Most often, the substrate is paper which can be a variety of weights and finishes. Other common substrates in commercial printing are films which include plastics, laminated plastics and foils. Process colors are printed using a combination of the four standard process inks: Cyan, Magenta, Yellow and Black (CMYK). Considering that every color used in printing press requires its own plate, it would be highly impractical to print using every color in a design. Process color printing was developed to address this impracticality, since most colors can be accurately printed with the use of these four colors. To create a process color which includes multiple inks, overprinting is used. Overprinting is the process of printing one color on

†The color matching functions (see Figure 1) are the average results of the color matching experiments which were conducted in the late 1920’s by David Wright and John Guild, for more details see.20 top of another in the reproduction of a design. Because of physical differences between the inks and substrate, the result of printing directly onto the substrate versus onto another ink may differ and needs to be considered. In some situations, it is necessary to print the desired color using a single ink or a spot color. Spot colors are particular premixed inks that are used instead of or in addition to process inks. In the print environment, each spot ink requires its own printing plate on the press. Spot colors are used instead of process colors for better color accuracy, better color consistency, colors outside of process ink gamut and for technologies which are prone to specific printing errors. The most common spot color system is PANTONE (http://www.pantone.com/). Similar to CMYK, it is usually possible to print a percentage of a given spot color. We refer to printing less than 100% of the spot as screening the spot or a spot tint. In this paper, there is also a need to use process colors to approximate the spot tint, process equivalent tint. The process equivalent tint would be the CMY percentages which produce the most accurate color to the target spot tint. 2.2.2 Print Limitations There are several printing limitations which may arise with any or all of the mentioned printing technologies, including: dot gain, minimum dot, registration and screening moiré patterns. The size of the printed dot may be affected by the ink, the substrate and the printing process. Dot gain or increase in halftone dot size occurs when wet ink spreads as it is absorbed by the substrate. The visual impact of dot gain is that the colors appear darker than expected. Some amount of dot gain is usually accounted for when the press is characterized, which is then stored within the color management workflow. The minimum dot or min dot is the smallest dot on a plate that can be consistently printed. It is highly dependent on the printing technology. For example, offset lithography (~2%) can typically hold a much smaller min dot than flexographic printing (~8%). With press printing a concern for the operator is registration. Substrates may stretch or shift due to changes in moisture and as the sheets are pulled through the press. These changes can cause multi-ink jobs to print out of register. The visual impact could be unexpected gaps or hue shifts between adjacent colors, the prints may also appear blurred. Another type of artifact which needs to be considered during the printing process is referred to as moiré pattern. These patterns may occur when two or more repeating patterns overlap each other. A screening moiré pattern is caused when two or more color plates are printed at screen angles less than 30° apart. The most severe pattern occurs when the screen angles between plates are very small. A set of standard screen angles for 4-color process color printing has been established. Yellow is the least visible, it’s placed at the most visible angle 0°. Black is the most visible color, so it’s placed at the least visible angle 45°. The Cyan and Magenta plates are placed at 15° and 75°, respectively.

2.3 Ink Overprint Models and Color Profiles Ink overprint models predict final color obtained by overprinting several inks on a specific press and substrate. These models will be used later by the embedding algorithm to predict (1) color of the overprint for visibility evaluation and (2) color of the overprint as seen by the imaging device for signal robustness evaluation. Due to narrow-band nature of the red LED of the POS scanner, models predicting full color spectrum are pre- ferred. Models predicting more compact color representation, such as CIE XYZ,7 can only be used for visibility evaluation. Ink overprint models are obtained in practice by combining two main factors (1) set of measured color patches printed on a real press and (2) mathematical model interpolating the measured values while making some simplifying assumptions. An ideal model would be obtained by measuring a set of color patches obtained by sampling the space of all possible ink combinations, possibly printed multiple times and averaged. For example, for k inks and n steps of each ink, nk color patches would have to be printed and measured. This process, known as press profiling or fingerprinting, is often used with process inks, where a few thousand patches are used to characterize the press. Measured values are then interpolated and assembled into k-dimensional look-up table which is further consumed by software tools. ICC profiles are standardized and industry-accepted form of such look-up tables converting k ink percentages into either CIE XYZ or CIELAB space. For process inks, 4-channel CMYK profiles are standardized to maintain consistency between different printers‡. Unfortunately, full color

‡GRACoL specification includes CMYK ICC profile recommended for commercial offset lithography (www. idealliance.org/specifications/gracol/). spectral data is often not available as standardization is still in progress.1 This methodology quickly becomes impractical as spot colors are introduced due to exponential increase of the number of patches required to print and large number of spot colors available. Popular mathematical model for ink overprint was described by Neugebauer.13, 17, 19 It expresses the spectral reflectance of a print as sum of the reflectance of each combination of ink (called Neugebauer primaries) weighted by the relative proportion of the paper it occupies. For example, for spot ink S, Cyan, and Magenta, we have

R(λ) = a0R0(λ)+aSRS(λ)+aC RC (λ)+aM RM (λ)+aSC RSC (λ)+aSM RSM (λ)+aCM RCM (λ)+aSCM RSCM (λ), (2) where R0(λ), RC (λ), RSC (λ) is reﬂectance of substrate, 100% Cyan ink, and overprint of 100% spot and Cyan all printed on substrate at wavelength λ, respectively. Other overprints, such as RSCM , are similarly deﬁned. Weights a satisfy Demichel equation17

a0 = (1 − αS)(1 − αC )(1 − αM ) aM = (1 − αS)(1 − αC )αM aCM = (1 − αS)αC αM

aS = αS(1 − αC )(1 − αM ) aSC = αSαC (1 − αM ) aSCM = αSαC αM , (3)

aC = (1 − αS)αC (1 − αM ) aSM = αS(1 − αC )αM where αS, αC , αM is spot, Cyan, Magenta ink percentage, respectively. The Spectral Neugebauer overprint model was enhanced by many authors, mainly by Yule and Neilsen,21 to describe more advanced effects of light penetration in the paper. For simplicity, we use the original Neugebauer model in this paper. In order to use the Spectral Neugebauer model with k inks in practice, we need to have reflectance of 2k Neugebauer primary colors including the color of the substrate, 100% of each ink on its own on the substrate, and all 100% ink overprint combinations printed on substrate. Reflectance of substrate, and any overprint of process inks can be derived (or at least approximated) from CIE XYZ values obtained from ICC profile. Reflectance of 100% of the spot color can be measured or taken from an external source such as PANTONE Live (www.pantone.com/live). Reflectance of multiple spot color overprint or process and spot ink overprint may be either measured from a printed test patch or, for transparent inks, approximated using product of reflectances. For example, reflectance of Cyan and spot overprint can be approximated by

RS(λ) RC (λ) RSC (λ) = R0(λ) . (4) R0(λ) R0(λ) Reflectance of process inks overprint can either be derived from an ICC profile CIE XYZ value or approximated as a product of individual reflectances normalized for substrate reflectance based on the formula above. When all inks are approximated by Eq. (4), we obtain k Y Ri(λ) R(λ) = R0(λ) 1 − 1 − αi . (5) R λ i=1 0( )

Coefficients αi in Spectral Neugebauer model are linear ink percentages before any dot gain correction. From Demichel equation (3), linear ramp in αi results in a linear change of reflectance and thus linear change of CIE XYZ. To correct for any single-ink non-linearity caused by the press (often called dot gain), we substitute αi in −1 −1 the above model with gain-corrected values gi (α î). Function gi inverts the dot-gain effect such that linear ramp in αî leads back to linear increase of reflectance. Several patches of single screened ink can be used to −1 estimate gi for i-th ink.

3. IMAGE CAPTURE DEVICE Watermark embedding algorithms slightly modify the original image intensities to find balance between visibility of the signal and robustness as seen by the reading device. Image capture device and its ability to resolve small changes thus play an important role in the overall process. Digimarc Barcode was designed to be readable by (1) modern imaging-based barcode scanners with red LED illumination and (2) smart phones equipped with a color camera. In the rest of this section, we review properties of both classes of the devices relevant for embedding. Colors with low Colors with high reflectance at 660nm reflectance at 660nm

Original color:

Scanner grayscale:

POS scanner visibility: visible transparent Figure 3. Selected colors and their grayscale representation as seen by a POS scanner with 660nm red LED light. Changes made to colors of low spectral reﬂectance at 660nm are visible (registered as black pixel value) to the scanner and thus are suitable for carrying watermark.

Modern barcode scanners are monochrome imaging-based devices§ equipped with a narrow-band red LED illumination. As specified by the GS1 standard [3, Section 5.5.2.7], the spectrum of the narrow-band illumination peaks at 670nm ±10nm range to be consistent with legacy laser-based scanners. The same wavelength is also recommended for devices grading readability of 1D and 2D codes. The particular POS scanner used in this paper has its peak spectrum at 660nm as shown in Figure 1. Due to the combination of a narrow-band illumination and monochromatic sensor, the barcode scanner can only see grayscale image created by spatial changes in ink reflectance at 660nm. If more inks are overprinted, the grayscale value G can be obtained from 660nm component of the Spectral Neugebauer model from Eq. (2) as G = sensitivity · R(660nm) + offset (6) for some sensitivity and offset constants. Based on spectral reflectance of process inks shown in Figure 1, the grayscale pixel value is mostly determined by the Cyan and Black inks. Due to close reflectance to the substrate, Yellow and Magenta inks appear transparent to the scanner. Spectral reflectance vary widely between spot inks. Figure 3 shows several color patches along with their simulated grayscale value to illustrate the visibility of different inks to the scanner. In this example, the black, navy blue, bright and dark purple patches all appear very dark where as yellow, pink and red patches all appear bright. This behavior of the scanner poses a unique challenge for embedding robust watermarks in spot colors. This is addressed in Section 5. Along with POS scanners, Digimarc Barcode can be read by smart phones running watermark reader app such as Digimarc Discover. In this case, full color image data is available and watermark signal is detected from grayscale image obtained by linear combination of RGB or YUV pixel values. This offers potential for embedding more robust watermark at the same visibility as will be discussed in Section 4. Other than (6), it is also of interest to support an additional grayscale conversion when embedding the watermark. In general, we have X G = sensitivity · w(λ)R(λ) + offset (7) λ for some grayscale conversion weight w.

4. WATERMARK EMBED IN CMYK ARTWORK In this section, watermark embedding is ﬁrst covered for a simpler case of process inks to introduce the concepts and discuss challenges. Section 5 then covers the general case of embedding into an arbitrary set of inks including spot colors. We illustrate the concepts with marked CMYK color swatches presented in Figure 5.

§Traditional barcode scanners are mostly laser based and capable of reading traditional linear barcode only (1D symbologies). These devices are now being replaced by 2D imager-based scanners capable of reading both 1D and 2D symbols such as QR and other matrix codes. 128

1 1 64 128 -1 0 +1 Figure 4. Grayscale representation of a single watermark tile printed at 75 watermark pixels per inch.

4.1 CMY Embedding Digimarc Barcode is a robust spread spectrum watermark.5 One instance carries a 47-bit payload, enough to encode exactly the same information as in Global Trade Item Number (GTIN-14)3¶ often carried by a linear barcode. Watermark carrying a specific payload can be represented, at spatial resolution of 75DPI, as a 128×128 grayscale image, called the watermark tile. Figure 4 shows one instance of a watermark tile at original scale with histogram of its values. Watermark tile is a zero-mean signal consisting of positive and negative values called waxels (watermark pixels). Watermark tiles can be concatenated next to each other and then cropped to cover larger area. As opposed to barcode, certain area of a single watermark tile can be cropped and still successfully decoded due to its repetitive structure. To some extend, watermark detection is also robust to image content into which the watermark tile was added. When print resolution is different from 75DPI, the watermark tile is up-sampled to match the print resolution. When embedding watermark tile into an artwork, one has to balance visibility of the signal with watermark robustness as observed by the user when scanning the final package. Based on the analysis of the scanner response under red illumination presented in Section 3, ink changes printed in Cyan or Black are visible to the scanner and thus can be used for printing the watermark tile. As discussed in Section 2.1, human visual system is significantly more sensitive to luminance changes (caused by changes in Black ink) than chrominance changes. For this reason, the watermark is embedded by modifying each of C, M, Y channels with grayscale values of the watermark tile W weighted by elements of the unit-length color weight vector ω = (ωC , ωM , ωY ) and global signal strength σ

0 0 0 Ci = Ci + σωC Wi,Mi = Mi + σωM Wi,Yi = Yi + σωY Wi, (8) where index i denotes the pixel of each color separation. Color weights ω drive the color of watermark signal, while σ changes the overall strength of the signal. Both parameters influence the visibility of the watermark. In general, the color weights depend on the ICC color profile associated with the CMYK artwork which captures the color of CMYK ink overprints. For a typical GRACoL profile, the color weights were set to ωGRACoL = (0.69, −0.61, 0.39). Even though the scanner does not see Magenta and Yellow changes, non-zero weights were chosen to minimize any luminance changes introduced by embedding the watermark signal in Cyan only. Figure 5 shows a sample color patches embedded with this method. To illustrate the effect of luminance cancellation and its effect on HVS, both Cyan-only and CMY chrominance watermark with the same Cyan component are shown. Due to spectral dependency of the red LED, only signal embedded in Cyan separation is available to the 2 detector. In case of ωGRACoL color weights, this only represents about 0.69 = 48% of the total signal energy embedded in the artwork that is extracted by the POS scanner. When full-color image sensor is available, such as in a smart phone, signal present in all CMY plates is combined by aligning the grayscale conversion weight w defined in Eq. (7) with ωGRACoL. In RGB color space, this grayscale conversion can be approximated as 0.52 · R − 0.81 · G + 0.29 · B. ¶GTIN-14 is a superset of codes containing all Universal Product Codes (UPCs) and International (European) Article Numbers EANs, all representable using common 14 decimal digit numbering scheme maintained by GS1 (www.gs1.org). Mid-gray CMY patch Compressed white - 4%C 2%M 2%Y

Chrominance watermark Cyan only Chrominance watermark Cyan only

Figure 5. Marked CMYK color patches illustrating diﬀerent embedding strategies discussed in Section 4.

4.2 Color and Press-Related Challenges When CMY ink combination is overprinted with Black to produce darker colors, the Black ink acts like an optical filter (consider approximation mentioned in Eq. (5)) and reduces magnitude of changes introduced in Cyan separation. This leads to weaker signal as seen by the scanner and thus the robustness of the watermark is degraded. This loss could be compensated either by increased signal strength σ, or by replacing portion of the Black ink with CMY combination making the final CMYK mix more suitable for watermarking, a process known as Under Color Addition. Colors with either no or 100% Cyan component pose another challenge. If Eq. (8) is applied blindly, half of the waxels will not be embedded due to clipping resulting in a robustness loss. This could be resolved by compressing the color gamut of the artwork. For example with no Cyan, 2%-4% Cyan ink can be added in the original design. Watermark is then inserted in this pre-conditioned artwork using methods described above. In order to utilize the full potential of Digimarc Barcode, 100% of the package surface should be watermarked. Some packages contain significant areas without any ink coverage. Such areas may lead to dead zones and reduce the full benefit of improved checkout speed. To resolve this, white areas can be covered by light CMY tint which is then used for watermarking. Tint consisting of 4%C, 2%M and 2%Y is recommended for offset printing and shown in Figure 5. Press characteristics such as dot gain and minimum dot need to be considered when watermarking a package design. Large minimum dot may limit the applicability of some of the solutions described above. Dot gain can usually be measured and compensated for by compressing positive and negative values of watermark tile differently before embedding in the artwork. Solutions to some of these challenges are currently being developed.

4.3 Robustness and Artwork Considerations When embedding a watermark in complex artwork, more signal can be embedded in textured areas than in flat regions because image texture masks the presence of the watermark signal. To better utilize this effect, both direction of the color weight vector and signal strength may vary spatially to equalize visibility of the watermark. This can be done either manually in image editing software or automatically as described in Ref. 14. Similarly, form factor of the package may require the signal strength to be increased. For example, the area of a soda can is significantly smaller than that of a cereal box. Larger objects provide more opportunities for the scanner to successfully read the watermark. Text overprinted on top of watermarked image background may also lead to robustness loss.

5. STRATEGIES FOR WATERMARKING SPOT COLOR ARTWORK In this section, a list of strategies is developed for watermarking consumer packages containing spot colors. Most consumer packages (say 75% of them) are printed with at least one spot color ink. Spot inks are often introduced as a way to achieve consistent color reproduction, or to produce colors outside process ink gamut. Some spot color inks provide special optical effects, such as fluorescence or metallic finish making the package unique. There are two main use cases of spot color inks in packaging. Traditionally, each extra spot ink represents a single color (printed either as solid or screened ink) and is usually not overprinted with other inks. A set of Spot only modulation CMY overprint modulation

Figure 6. Marked spot color patches illustrating different embedding strategies discussed in Section 5. common brand colors is often shared within similar products. This option is historically convenient, due to lesser requirements on press tolerances. It is easier to print a single ink than to maintain color consistency of several inks overprinted on top of each other. Slight miss-alignment of printing plates and ink adhesion could contribute to possible color shift from the original color if the press is not controlled properly. More recently as the press tolerances have improved, spot color inks are being treated like process inks and overprinted together to extend the color gamut of the press. For example, Hexachrome printing process uses CMYK inks along with orange and green. For some jobs, this technology offers the advantage of being able to print significantly more brand colors with reduced number of printing plates. With spot colors, several new challenges appear for watermarking. If the spot color is used as solid color, embedding the watermark tile directly in this spot color does not produce good results. This is either because the ink may not be of low enough reflectivity at 660nm and thus the watermark will not be visible to the POS scanner, or direct spot color modulation may lead to severe visual artifacts. This effect is shown in Figure 6, where solid patch of PANTONE 221 C purple color of 35% spectral reflectance is shown. Due to the spectral reflectance at 660nm, only about 35% of the signal is extracted by the detector if spot color is modulated directly. Certain colors, such as yellow, have very high spectral reflectance at 660nm and thus are not seen by the POS scanner at all. One option for watermarking spot color is to replace the spot color ink with its closest CMYK equivalent. This is often not acceptable solution since the spot color was introduced in the artwork by the designer for a reason. In case of metallic colors, the special effects would be completely lost.

5.1 Spot Color Embed by Ink Overprint Similarly as with watermarking white substrate using light CMY tint described in Section 4.2, we address the problem of watermarking spot colors by overprinting the spot ink with light tint composed of other inks available in the design. In designs where process colors are available, CMY tint is used. This process is demonstrated in Figure 7 on a case of watermarking 100% PANTONE 221 C purple color with CMY overprint. First, we screen the spot color back to 75% and approximate the remaining 25% with CMY inks. By doing this, we make ourselves room for future CMY tweaks. The color difference between 100% spot and new 75% with overprinted CMY tint, denoted as ECM, is called the Color Match Error and can be measured using any color difference metric mentioned in Section 2.1. Next, we decompose the CMY tint into so-called Max and Min Tweaks by minimizing weighted color difference between them while achieving required difference in spectral reflectance at 660nm when overprinted with 75% spot. Watermark is embedded into CMY tint by interpolating between min and max tweaks based on watermark tile values, i.e., TCMY = TMin +(1+W )(TMax −TMin)/2. Color difference between min and max tweaks overprinted with 75% spot, denoted EWM, is called the Watermark Error. Final watermark is produced by overprinting 75% screened spot and marked CMY tint. In this process, both color errors are interconnected. In order to keep luminance changes minimal more space for CMY tweaks may be needed and thus possibly increasing the color match error. Spot screen of 75% is also a parameter that could be changed. Difference of spectral reflectance at 660nm, denoted as ∆660, serves as measure of watermark signal strength similar to parameter σ in Eq. (8).

Given the required value of ∆660, spectral ink overprint models can be used to ﬁnd optimal value of spot 75% screened spot

Fixed ink path (spot inks)

simulate Watermark tile overprint E overprint actual CM Max tweak Unmarked digitally inks Marked PANTONE 221 PANTONE 221

Optimized ink path (CMY inks) EWM interpolate tweaks

CMY tint approximates Watermarked CMY tint 25% of spot Min tweak Figure 7. Spot color embed workflow with CMY overprint described in Section 5.1. Ink overprint is simulated for evaluation of color match error ECM and watermark visibility EWM . Only the first one is shown in the figure. screen and min and max tweak ink percentages minimizing weighted sum of both color errors

2 Rmax + Rmin 2 min ECM + p · EWM = ∆E76 RS, + p · ∆EWM (Rmin,Rmax) , (9) αmin,αmax 2 s.t. Rmax(660) − Rmin(660) ≥ ∆660

0 ≤ αmin, αmax ≤ 1 where Rmax and Rmin correspond to Neugebauer spectral reflectance from Eq. (2) obtained for (spot and CMY) ink percentages αmax and αmin, respectively. RS refers to spectral reflectance of the original spot color printed on substrate. Color difference metrics ∆E76 and ∆EWM are described in Section 2.1. Both metrics return scalar values weighted by a constant penalty term p. In general, weight p is dependent on the color. From experiments conducted with professional designers, we set the default value of the weight factor to p = 1. The optimization problem in Eq. (9) was solved numerically using IPOpt library.18 By formulating the embedding as an optimization problem, other press or design-related constraints can be put in place. For example, designers may not allow the spot ink to be screened due to physical press reasons. By including the spot ink in αmin and αmax without any constraint will allow the spot ink to be modulated by the watermark tile. In this case, a specific spot ink is moved from the fixed-ink path in Figure 7 to the optimization path and is included in the optimization process along with CMY inks. Min and max tweaks then contain spot ink component and watermark tile is embedded by interpolating between these two colors exactly as described in previous paragraph for the CMY case. This formulation of the embedding problem is not limited to spot ink being overprinted by CMY inks. The same formulation can be used with any set of inks that could be overprinted in the package design. For example, two spot colors could be used to embed chrominance watermark. This technique is necessary for watermarking spot colors used in Extended Gamut printing process such as Hexachrome. Constraints related to additional grayscale conversion weights from Eq. (7) can also be added to consider signal strength as seen by full-color devices such as mobile phones. 5.2 Color and Press-Related Challenges Color and press-related challenges with watermarking process inks mentioned in Section 4.2 carry over to spot colors. For example, large minimum dot on some presses may require new solutions to be developed. Some colors, such as dark blue or bright green have very low reflectance at 660nm and all appear as black to the POS scanner. Any signal embedded in CMY inks overprinted on top of such colors will be filtered out and not visible (consider approximation in Eq. (5)). Such inks may have to be either screened down to allow CMY overprint changes to be visible or included in the optimized ink path in Figure 7 to allow modulation by watermark tile. Consider a spot color very similar to Cyan ink. In this case, the cyanish spot color should be modulated instead of introducing another Cyan ink overprint. As mentioned in Section 2.2, assigning the best screening angles to each color plate is important for reducing the impact of moiré patterns. Typically, when using a screened spot color with 4-color process colors, use the screen angle of the least prominent or missing screened process color. In our workflow, this would be Black, since our process tints usually only include CMY. When overprinting two screened spot colors with CMY, the best screen angle for each spot will depend on the spot colors, but will likely be the Magenta or Cyan angles. Yellow should be avoided because it is only 15° from Cyan and/or Magenta. As a result moiré is always there but usually not visible since Yellow is so light.

6. CONCLUSION An automated method has been described for watermarking a range of spot colors printed on different substrates using offset and flexographic printing technologies. The Digimarc Barcode is inserted in the printing ink domain, using an Adobe Photoshop plug-in as the last step before printing. Since Photoshop is an industry standard widely used by pre-press shops in the packaging industry, a Digimarc Barcode can be easily inserted either internally by the Professional Services Group at Digimarc or externally by a pre-press partner.16 Since the traditional barcode does not have to be presented to the POS scanner, this significantly improves the IPM metric, which retailers use to track the checkout efficiency since it closely relates to their profitability. Increasing IPM by a few percent could lead to potential savings of millions of dollars for retailers, giving them a strong incentive to add the Digimarc Barcode to their packages. An increase in IPM of at least 33% has been measured for a typical range of packages watermarked in this way.8 In the future we plan to add the ability to estimate the robustness of Digimarc Barcode digitally, when the watermark is being applied to an image, and/or with a custom hardware reader using a printed Digimarc Barcode in a similar manner to grading a traditional barcode.2

ACKNOWLEDGMENTS Don Haaga from the Digimarc Professional Services Group developed a suitable light tint for use in white package areas. The CMY tint value is designed to appear white to the HVS but carry a robust Digimarc Barcode. The authors would also like to thank Joel Meyer and Don Haaga of Digimarc Corporation for editing the manuscript.

REFERENCES 1. International Color Consortium, iccMAX. See http://www.color.org/iccmax.xalter. 2. GS1 Association. 1D Barcode Verification Process Implementation Guideline. Version 24, June 2014. Available at http://www.gs1.org/docs/barcodes/GS1_Bar_Code_Verification.pdf. 3. GS1 Association. GS1 General Specifications. Version 14, January 2014. Available at http://www.gs1. org/genspecs. 4. Digimarc Corporation. Digimarc Barcode: A quantitative model of annual labor cost savings and return on investment for retailers. January 2014. Available at http://model.digimarc.com. 5. I. J. Cox, M. L. Miller, J. A. Bloom, J. Fridrich, and T. Kalker. Digital Watermarking and Steganography. Morgan Kaufman Publishers Inc., San Francisco, CA, 2007. 6. G. J. Van der Horst and M. Bouman. Spatiotemporal chromaticity discrimination. Journal of Optical Society of America, 59, 1969. 7. K. Deshpande and P. Green. A simplified method of predicting the colorimetry of spot color overprints. Color and Imaging Conference, pages 213–216, 2010. 8. B. Gerlach, D. T. Pinard, M. Weaver, and A. M. Alattar. Scanning-time evaluation of Digimarc Barcode. In A. M. Alattar, N. D. Memon, and C. D. Heitzenrater, editors, Proceedings SPIE 9409, Media Watermarking, Security, and Forensics 2015, volume 9409, February 2015. 9. H. Kipphan. Handbook of Print Media. Springer, 2001. 10. R. Lyons, A. Reed, and J. Stach. Geometric enumerated chrominance watermark embed for spot colors. In Q. Lin, J. P. Allebach, and Z. Fan, editors, Proceedings SPIE 8664, Imaging and Printing in a Web 2.0 World IV, volume 8664, March 2013. 11. F. L. Van Nes and M. A. Bouman. Spatial modulation transfer in the human eye. Journal of Optical Society of America, 57(3):401–406, Mar 1967. 12. A. NeSmith. Differences between flexographic, gravure, offset and inkjet printing. Oren International, 2013. Available at http://www.oren-intl.com/blog/bid/301939/ Differences-Between-Flexographic-Gravure-Offset-and-Inkjet-Printing. 13. H. E. J. Neugebauer. Die theoretischen grundlagen des mehrfarbenbuchdrucks. Zeitschrift Wissenschaften Photography, 36:73–89, 1937. 14. A. Reed, D. Berfanger, Y. Bai, and K. Falkenstern. Full-color visibility model using CSF which varies spatially with local luminance. In Q. Lin, J. P. Allebach, and Z. Fan, editors, Proceedings SPIE 9027, Imaging and Multimedia Analytics in a Web and Mobile World 2014, volume 9027, 2014. 15. R. L. Renesse. Optical Document Security. Artech House Optoelectronics Library, 3rd edition, 2004. 16. T. Rodriguez, D. Haaga, and S. Calhoon. Automation and workflow considerations for embedding Digimarc Barcodes at scale. In A. M. Alattar, N. D. Memon, and C. D. Heitzenrater, editors, Proceedings SPIE 9409, Media Watermarking, Security, and Forensics 2015, volume 9409, February 2015. 17. J. A. S. Viggiano. Modeling the color of multi-colored halftones. TAGA proceedings, 42, 1990. 18. A. Wächter and L. T. Biegler. On the implementation of a primal-dual interior point filter line search algorithm for large-scale nonlinear programming. Mathematical Programming, 106(1):25–57, 2006. 19. D. Wyble and R. S. Berns. A critical review of spectral models applied to binary color printing. Color Research & Application, 25(1):4–19, 2000. 20. G. Wyszecki and W. Stiles. Color science: concepts and methods, quantitative data, and formulae. John Wiley & Sons, 2000. 21. J. A. C. Yule and W. J. Neilsen. The penetration of light into paper and its effect on halftone reproduction. TAGA proceedings, pages 65–76, 1951. 22. X. Zhang and B. Wandell. S-CIELAB: a spatial extension to the CIE L* a* b* DeltaE color difference metric. 1998. See http://white.stanford.edu/~brian/scielab/scielab.html.