sign in sign up
<<
Home , Film speed

US007515733B2

(12) United States Patent (10) Patent No.: US 7,515,733 B2 Rhoads (45) Date of Patent: Apr. 7, 2009

(54) METHODS AND ARRANGEMENTS 5,134,719 A 7, 1992 Mankovitz EMPLOYING DIGITAL CONTENT TEMIS 5,210,820 A 5/1993 Kenyon 5,577,266 A 11/1996 Takahisa et al. (75) Inventor: Geoffrey B. Rhoads, West Linn, OR 5,606,609 A * 2/1997 Houser et al...... 713/179 (US) 5,646,997 A 7/1997 Barton 5,664,018 A 9/1997 Leighton (73) Assignee: Digimarc Corporation, Beaverton, OR 5,703,795 A 12/1997 Mankovitz (US) (*) Notice: Subject to any disclaimer, the term of this (Continued) patent is extended or adjusted under 35 U.S.C. 154(b) by 742 days. FOREIGN PATENT DOCUMENTS (21) Appl. No.: 11/039,720 WO WO9514289 5, 1995 (22) Filed: Jan. 19, 2005 OTHER PUBLICATIONS (65) Prior Publication Data “Frequently Asked Questions About Digimarc Signature Technol US 2005/O196O13 A1 Sep. 8, 2005 ogy.” Digimarc Corporation, Aug. 1995, 9 pages. Related U.S. Application Data (Continued) (63) Continuation of application No. 10/821,414, filed on Primary Examiner—Jose L Couso Apr. 9, 2004, now Pat. No. 7,035,427, which is a con tinuation of application No. 09/545,174, filed on Apr. (57) ABSTRACT 6, 2000, now Pat. No. 6,775,392, which is a continua is epit 613. filed on Nov. 12, Methods and arrangements for identifying content, and s • L vs - Ys a 1-1-9 s - - employing such identification, are detailed. One method (51) Int. Cl. embeds a plural-bit digital watermark into content, but first H04LK LM00 (2006.01) checks to see if the content is ppreviously y watermarked. (52) U.S. Cl 382/100 Another method applies a digital watermark detection proce ir grgrrr. dure to only a sub-portion of a digital content item. Yet (58) Field of sstateStars 5.455.200 3.s another arrangement involves plural-portion content, where 380/217: 705/50 57 59. 707/ 9. 1316 one portion is watermarked with first data governing its rights s 71 3/i 7 6 179: , 15 f515 51 6 management, and another portion is watermarked with sec See application file for complete search histo ond data governing its rights management. Still another pp p ry. method concerns distribution of content items, where each is (56) References Cited watermarked with a unique ID as part of the distribution process. Yet another method concerns deriving an identifier U.S. PATENT DOCUMENTS from content, and using the content to access related metadata 4,230,990 A 10/1980 Lert, Jr. et al. from a remote computer system. A variety of other technolo 4,528,588 A 7/1985 Lofberg gies and improvements are also detailed. 4,547,804 A 10/1985 Greenberg 4,590,366 A 5/1986 Rothfjell 5 Claims, 51 Drawing Sheets

Create an image Use an image

Create and Ced Open an Image Watermark Distribute Digitally © Icon Appears an image 3 or in Print 3: Leads to Attributes and Creator ID . Get a look up Adobe Photoshop 4.0 Creator D Creator Adobe Photoshop 4.0 (One time) information 35% PictureMarc

MarcCentre Locator Service (Internet and fax) US 7,515,733 B2 Page 2

U.S. PATENT DOCUMENTS 2002fOO59238 A1 5, 2002 Saito 2003, OO11684 A1 1/2003 Narayanaswami et al. 5,719,937 2, 1998 Warren et al. 5,721,788 2, 1998 Powell OTHER PUBLICATIONS 5,761,606 6, 1998 Wolzien 5,765, 176 6, 1998 Bloomberg Cox et al. “Secure Spread Spectrum Watermarking for Multimedia.” 5,772,250 6, 1998 Gasper NEC Technical Report 95-10, Jan. 1995. 5,809,139 9, 1998 Girod Digimarc, Frequently Asked Questions About Digimarc Signature 5,822.432 10, 1998 Moskowitz et al. Technology, 9pp., 1995. 5,822,436 10, 1998 Rhoads Highwater FBI, Copyright Protection for Digital Images, Digital 5,825,892 10, 1998 Braudaway Fingerprinting from FBI, 4 page brochure, 1995. 5,859,920 1, 1999 Daly et al. Matsutani. The Construction of Copyright-Protected Image Data 5,901,178 5, 1999 Lee et al. Technical Report of IEICE, ISEC94-58, 1995. 5,905,800 5, 1999 Moskowitz et al. van Schyndel, Towards a Robust Digital Watermark, Asian 5,910,987 6, 1999 Ginter Conference on Computer Vision, pp. 504-508, 1995. 5,919,730 7, 1999 Gasper et al. Zhao, A WWW Service to Embed and Prove Digital Copyright 5,987,127 11, 1999 Ikenoue et al. Watermarks, Proc. Of the European Conf. On Multimedia, Services 6,111,954 8, 2000 Rhoads and Techniques, pp. 1-15, May 1996. 6,130,741 10, 2000 Wen et al. Zhao, Embedding Robust Labels into Images for Copyright Protec 6,438,231 8, 2002 Rhoads tion, Proc. of the International Congress on Intellectual Property 6,751,320 6, 2004 Rhoads Rights for Specialized Information, Knowledge and New Technolo 6,931,451 8, 2005 Logan et al. gies, 10 pages, 1995. 7,362,775 4, 2008 Moskowitz 2002fOO48387 4, 2002 Rhoads * cited by examiner U.S. Patent Apr. 7, 2009 Sheet 1 of 51 US 7,515,733 B2

FG. 4 COMPUTER

1 O6

OO

EXPOSE AND STEP

'4-BITS

|LLLLLLLLL)5|-||||). È||||52 ||||||E? ||||||==||||||? +-- U.S. Patent Apr. 7, 2009 Sheet 2 of 51 US 7,515,733 B2

OBAN OR CREATE ORIGINAL FG 2 DGA SIGNAL OR MAGE

ESMA E ROUGH OFFSET AND RMS NOISE

CHOOSE N OR N-BT DENTIFICATION WORD, E. G. 32 GENERATE N-BIT DENTF CATION WORD

GENERATE OR SYNTHESIZE N "RANDOM" NDEPENDENT SIGNALS WITH ROUGHLY GAUSSIAN DISTRIBUTION ABOUT SOME MEAN VALUE WHERE SIGNALS HAVE EQUAL EXTENT AND D, G|A SPACNG OF ORIGINAL DIGITAL SIGNAL OF IMAGE

APPLY OGAL FILTER WHICHATTENUATES BOTH LOW AND HIGH FRE OUENCES, LEAVING MDOLE-RANGE FRE OUENCIES ARGELY NTACT

CONDENSE N RANDOM SIGNALS TO A OWEST ACCEPTABLE BIT WALU S ; F MEMORY OR SORAGE SPACE SAF A PREM UNA

AOD ALL RANDOM MAGES TO GE HER WHEC- HAVE A CORRESPONDING '1' N HER ASSOCATED BT-PLACE-VALUE OF THEN-BIT DEP.JTFICATION WORD, CALL THIS THE BASE COMPOSITE SIGNAL OR MAGE

EXPERMEN VISUALLY WITH GAN AND APLIED TO BASE COMPOSITE SIGNAL OR MAGE, ADDING THS TO ORIGINAL DIGITAL SIGNAL OR IMAGE AND DETERMINING THE ACCEPTABLE PERCEVED NOSE LEVEL

APPLY FOUND GAN AND GAMMA TO BASE COMPOSITE, ADD TO ORIGINAL THEN CALL THIS THE DISTRIBUAB ESGNA OR MAGE

STORE AWAY AND SECURE ORIG MAL SIGNAL OR IMAGE, ALONG WITH N-BT DENTIFICATION WORD AND THEN RANDOM SiGNALS

SE OR DISTRIBUTE THE DISTREUTABLE SIGNAL OR MAGE U.S. Patent Apr. 7, 2009 Sheet 3 of 51 US 7,515,733 B2

OBTAN OGITAL OR NON-DGITAL COPY FG 3 OF SUSPEC SGNAL OR MAGE DGTZE F NOT AL READY DIGITAL

CUT AND MASK PORTION OF SIGNAL OR MAGE BE, EVEO TO BE SUSPECT (ONLY FENTRE SIGNAL OR IMAGE IS NOT SUSPECT) PROCURE ORIGINAL DIGITAL SIGNAL OR MAGE AND CU AND MASK TO ROUGHLY THE SAME LOCATION OR SEOUENCE

VISUALLY FRESCALE AND REGISTER THE CU-OUT SUSPEC SIGNAL TO THE CUT-OUT ORIGINA SIGNAL

RUN THROUGH SEARCH PROGRAM WITH MEAN SQUARED ERROR AS CRITERIA AND X OFFSET, Y OFFSET, AND SCALE AS THE THREE WAR AEUES APPLY X OFFSET, Y OFFSET, AND SCALE TO CUT-OUT SUSPECT, THEN RESAMPLE ONTO EXACT GRD AND CUT-OF. T

OF ORIGINAL SGNAL

RUN THROUGH SEARCH PROGRAM WITH MEAN SOUAREO ERROR AS CRETERIA AND DC OFFSET, GAN, AND GAMMA AS THE HREE WARIABLES: APPLY TO SUSPECT SUBTRACTORIGINAL FROM SUSPEC GWNG OFFERENCE SIGNAL OR MAGE

STEP THROUGH ALL N RANDOMINDEPENDENT SIGNALS, MASKED AS ORIGINAL AND CROSS-CORRELATED WITH OFFFERENCESGNA N M MEDATE NEGHBORHOOD OF REGISTRA iON PONS

FIND OAND LEVEL BY AVERAGING FRST FOUR 01 01 CODEVALUES ASSGN E HERA O OR A 1 TO EACH CROSS-CORRELATION RESU DEPENDING ON PROX MTY TO THE AVERAGES OF PREVIOUS STEP CHECK RESULT AGAINST SECURED DENTIFICATION NUMBER PROSECUTE FIT MATCHES2 OR AT LEAST SENO A NASTY LETER DEMANDING RECOMPENSE U.S. Patent Apr. 7, 2009 Sheet 4 of 51 US 7,515,733 B2

FG 5 DENT FICATION CODE WORD -- CODED OUTPUT (e.g. 01101001) REA-ME SGNAL ENCOOER NPUT --- KEY DATA SGNAL (OPTIONAL)

FG 6 ------ANA OG t 222 NOSE 202 SOURCE

24

OO 1 OOO N-216 U.S. Patent Apr. 7, 2009 Sheet 5 Of 51 US 7,515,733 B2

NOSE SOURCE

2O6

REAL INPUT TME OUTPUT ENCODER

O O O O O O O O O 1 O O O O 1 O O O O O O ST D-O O O O O 1 THROUGH

O99 O 83O O 1 89O / NTHWORDS CODE O O 1 O O 1 1 FG 7

FG 8 248 REAL 2O2 -1

TME ENCODER OUTPUT

A NOISE A CODE

NOSE SOURCE O O O O O O O O 0 1 0 OO 1 O

RESET O O O O O SYNC O O O O O O DETECTOR NCREMENT U.S. Patent Apr. 7, 2009 Sheet 6 of 51 US 7,515,733 B2

OO O SECONDS FG 9 BNORMALIZEDMVAWNNY 402 -2OdB FREGUENCY -40dB

OHZ 5OKHZ

FG 9 C |O O.O REPEATED BORDER CONTINUITY 4O4

DETAL OF MATCHAT BORDER: CONTNUOUS TO mth DEFR WITIVE

ROM 504 STANDARD NOSE SGNATURES AUDIO COPYRIGHT

N DETECTION FLAG 7 5O2 5 OO FLAG WAD

506 U.S. Patent Apr. 7, 2009 Sheet 7 Of 51 US 7,515,733 B2

SANDARD NOSE 504 SIGNATURE

READ OUT AT25% - 604 NORMAL RATE

PTCH CHANG ED - 606 NOISE SIGNA Y -- SIGNAL-PITCH CANCELLED 608 NOSE SIGNAL

y TMEAVERAGED 6 O TMEAVERAGED POWER SIGNAL P.C. NOISE POWER SIGNAL SIGNAL Ps-PCN y POWER DIFFERENCE SIGNAL s-PCN FSG 6UT -- 6263 : - 64-CASE 1: nWvuln/WMNNNNNNNNrivumI - T - OS 5S i0s 15s ---CASEV616 V - 2: I - ^- 4 SECOND BEATS N-618 FIG. 12 624 NAUDIO VALID FLAG VALID COUNTER SIGNALN- 508

SIGNAL ONE .05S DIGITAL DELAY

62O .05s DIGITAL DELAY 5O2 .05s DIGITAL DELAY DEECTION -- FLAG U.S. Patent Apr. 7, 2009 Sheet 8 of 51 US 7,515,733 B2

F.G. 13

MOVE BUD'S ADVENTURES

7 OO

FRAME 1283 DSTRIBUTON E O REGON 4

ENCRYPON/SCAMBING ROUTINE #28, 7O2

704

PSEUDO-RANDOM MASTER SNOWY MAGE (SCALED DOWN AND ADDED TO FRAME 12783) U.S. Patent Apr. 7, 2009 Sheet 9 of 51 US 7,515,733 B2

FG. 4

- EMBEDDED 720s

DFFERENCE V w

MEAN-REMOVED HISTOGRAMS OF OFFERENCE SIGNAL AND KNOWN EMBEDDED CODE SIGNAL

- EMBEDDED

DIFFERENCE v Y N THRESHOLDING

MEAN-REMOVED HISTOGRAMS OF FIRST DERVATIVES (ORSCALERGRADIENTS IN CASE OF ANIMAGE) U.S. Patent Apr. 7, 2009 Sheet 10 of 51 US 7,515,733 B2

STEP Z FRAMES 750

RENDER LOGO FRAME 7OO ENCRYPT/SCAMBLE 702 FG 15 : OUTPUT MASTER SNOWY 704 MAGE FRAME

HGH BRIGHTNESS MASTER 752 SNOWY MOVE 754. . 758

-- "CHEAP MASTER COMPRESSED MASTER . . . . . SNOWY MOVE ORIGINAL MOVE

766

77O 762

DISTRBUTABLE SDE-BY-SIDE ORIGINAL MOVE MOVE VIEWING

------Apr. 7, 2009 Sheet 11 of 51 US 7,515,733 B2

U.S. Patent Apr. 7, 2009 Sheet 12 of 51 US 7,515,733 B2

FG 17 96 BT LEADER STRING, 820 "SHADOW CHANNEL", 828 M -/- - . .. 64. BIT LENGTH | 32 BT DATA WORD SIZE DATA 822 u) 824 826 UNIVERSA EMP RCA DATA FORMAT

F.G. 18

852 1.

864 1 U.S. Patent Apr. 7, 2009 Sheet 13 of 51 US 7,515,733 B2

F.G. 19 ×

O EST FOR MOSAICEO KNOT PATERNS WHICH "COVER" AND A E COEXTENSIVE WITHORIGINAL IMAGE, A L ELEMENTAL KNOT PATTERNS CAN CONVEY THE SAME 2 FORMATION, SUCH ASA SIGNATURE OR EACH CAN CONVEYA N W MESSAGE IN A STEGANOGRAPHC SENSE U.S. Patent Apr. 7, 2009 Sheet 14 of 51 US 7,515,733 B2

874

FG 20 CENTERPONT OF RING, 872

- - wr w NOMNAL DISANCE NOMNA)SAMCE O CENTER OF OUTER O CENTER OF OUTER RING WIDTH, 870 RING WIDTH, 870

2-D BRIGHTNESS OF PHASE. ONLY FILERED RNG IS SMLAR TO THE ABOVE BRIGHTNESS PATERN ROATED ABOUT CENTRAL POINT OF RING:

U.S. Patent Apr. 7, 2009 Sheet 15 of 51 US 7,515,733 B2

F.G. 2A

900- WHERE C - 1/16

ELEMENTARY BUMP (DEFINED GROUPING OF WITH WEIGHT VALUES)

| | | | | | c.2cc 4 6 7 to 1 2 3 || || || || EXAMPLE OF HOW MANY ELEMENTARY BUMPS, 900, WOULD BE ASSIGNED LOCATIONS IN AN MAGE, AND THOSE LOCATIONSWOULD BE ASSOCIATED WITH A CORRESPONDING BIT PLANE IN THEN-BIT WORD, HERE TAKEN ASN-8 WITH INDEXES OF O-7. ONE LOCATION, ASSOCIATED WITH BT PLANE"5". HAS THE OVERLAY OF THE BUMP PROFILE DEPICTED. U.S. Patent Apr. 7, 2009 Sheet 16 of 51 US 7,515,733 B2

DGITAL MAGE TAKEN OF CUSTOMER

COMPUTER PROCESSES NEW ENCODEO MAGE PRINTS

952 PROCESSED

MAGE ONTO PLASTC CARD

E. li?till it 2. A. 954

U.S. Patent Apr. 7, 2009 Sheet 17 of 51 US 7,515,733 B2

FG 23 CABLE, 964, TO DATA LINE, 966

96.O

958

CONTAINS RUDIMENTARY OPTICAL SCANNER, MEMORY BUFFERS, COMMUNICATIONS DEVICES, AND MCRO PROCESSOR

CONSUMER MERELY PLACES CARD INTO WINDOW AND CAN, AT THEIR PREARRANGED OPTION, ETHER TYPE IN A PERSONAL DENTIFICATION NUMBER (PN, FOR ADDED SECURITY) OR NOT. THE TRANSACTION IS APPROVED OR DSAPPROVED WITH IN SECONDS U.S. Patent Apr. 7, 2009 Sheet 18 of 51 US 7,515,733 B2

FG 24

ORIGINAL WITH COMPUTER GENERATES MASTER BARCODE AND FIDUCAS SNOWY MAGE WHICH IS ADDED GENERALLY OR HOGONALTO ORIGINAL MAGE AT LEFT

i. iii. Ur

COMBINED TO FORMPERSONAL CASH CARD

U.S. Patent Apr. 7, 2009 Sheet 20 of 51 US 7,515,733 B2

FG 26 THE NEGLIGBLE-FRAUD CASH CARD SYSTEM

982 1 O

O

CENTRA O NETWORK 980 O O 984 O

C o O

O O C O

O O

A BASC FOUNDATION OF THE CASH CARD SYSTEM S A 24-HOUR INFORMATION NETWORK, WHERE BOTH THE STATIONS WHICH CREATE THE PHYSICAL CASH CARDS, 950, AND THE POINT-OF-SALES, 984, ARE ALL HOOKED UP O THE SAME NETWORK CONTNUOUSLY U.S. Patent Apr. 7, 2009 Sheet 21 of 51 US 7,515,733 B2

OO2

WEBSITE DEVELOPMENT TOOL WWW SITE

OO6

O1 O BROWSER WWW STE

FG. 28 1010 1000

O 12

CITIZENSHIP U.S. Patent Apr. 7, 2009 Sheet 22 of 51 US 7,515,733 B2

FG 27A U.S. Patent Apr. 7, 2009 Sheet 23 of 51 US 7,515,733 B2

FG 29 UV PLANE, 1 OOO EMBEDDEO S GNAL POWER SPECTRUM, . OO2 ---. ... TYPICAL POWER SPECRUM es - OF ARBITRARYOO4 IMAGE,

NON-HARMONIC SPATAL FREOUENCES ALONG THE 45 DEGREE AXES, GIVING RISE TO AWEAVE-LIKE CROSS-HATCHING PA ERN IN THE SPATAL DOMAIN

FG 3O UV PLANE, OOO

EMBEDDED SIGNAL POWER SPECTRUM, 1 OO6 wa-e- YPCAL POWER SPECTRUM OF ARBETRARY MAGE, OO4

NON-HARMONC CONCENTRC CIRCLES 1N UV PLANE, WHERE PHASE HOPS OUAS-RANDOMLY ALONG EACH CIRCLE GIVING RISE TOPSEUDO RANDOM LOOKING PA ERNS IN THE SPATAL DOMAIN U.S. Patent Apr. 7, 2009 Sheet 24 of 51 US 7,515,733 B2

H SPATIALTRANSFORM DOMAI

E. w

S.

H IIT III

w Y HHHHH EHs ... ITT T. NIIT III ITT

is II itHH is

TT H

H . . . .

TTTE IT I ITT s TTTTTT T. III TTT

.

U.S. Patent Apr. 7, 2009 Sheet 25 Of 51 US 7,515,733 B2

FG 31A

PI PHASE OF SPATIAL FRECRUENCES ALONG FORWARD 45 DEGREE AXES, O08

PHASE OF SPATA FREGUENCES ALONG BACKWARD 45 DEGREE AXES, 1010

PHASE OF SPATAL FREOUENCES ALONG FIRST CONCENTRC RING, 1 O2

PHASE OF SPATAL FRECRUENCEES ALONG O SECOND CONCENTRC RING, O4. -PI FG 32C P N-1 Y-FI PHASEFESESSESAbng OF SPATIAL O THIRD CONCENTRC RING, 1 O16 -P - \ U.S. Patent Apr. 7, 2009 Sheet 26 of 51 US 7,515,733 B2

FG 33A FG 33B

ANGLE A ANGLEB

FG 33C THR O

POWER PROFILE ALONG ANGLE A, AS NORMALIZED BY ITS OWN MOVING AVERAGE ONLY A MINIMAL AMOUNT EXCEEDS THRESHOLD, GIVING ASMALL INTEGRATED VALUE

POWER PROFILE ALONG ANGLE B, AS NORMALIZED BY ITS OWN MOVING AVERAGE: THIS FINDS STRONG ENERGY ABOVE THE THRESHOD

O26 AS ANGLE IS SWEP FROM FG 33E \ OHIGHEST TO 90, 45 ENERGYSTHREHOLD DEGREES HAS

O DEGREES 45 DEGREES 9 O DEGREES U.S. Patent Apr. 7, 2009 Sheet 27 of 51 US 7,515,733 B2

F.G. 34A

RAO US B

O

O34 THR O

POWER PROFI EAONG CIRCLE AT RADUS B

FG 34E NTEGRATED / POWER THRESH TOTAL INTEGRATED POWER N ABOVE THRESHOLD AS 1040 FUNCION OF RADIUS U.S. Patent Apr. 7, 2009 Sheet 28 of 51 US 7,515,733 B2

FG 35A

SCALE = A; ADD ALL POWER WALUES AT THE "KNOWN" FREQUENCIES", 1042 FG. 35B

SCALE = B; ADD ALL POWER VALUES AT THE "KNOWN" FREQUENCIES", 1044

FG 35C

25% 1 OO% 4OO SCALE 50%, 2OO% SCALE "SCALED-KERNEL" BASED MATCHED FILTER; PEAK S WHERE THE SCALE OF THE SUBLMINAGRD WAS FOUND, 1046 U.S. Patent Apr. 7, 2009 Sheet 29 of 51 US 7,515,733 B2

Y22. :

ARBTRARY ORIGINAL MAGE NWHICH SUBLMINAL GRATICULES MAY HAVE BEEN PLACED "COLUMNSCAN" SAPPEED ALONGA GIVEN ANGLE THROUGH THE CENTER OF THE IMAGE

COLUMNINTEGRATED 056 FG. 36C GREY 1 1058 VALUES, 1054 M START OF END OF SCAN SCAN

po FG. 36D

MAGNITUDE OF FOURIER TRANSFORM OF SCAN DATA U.S. Patent Apr. 7, 2009 Sheet 30 of 51 US 7,515,733 B2

FIG 37 PROCESS STEPS

1. SCAN IN 2D FF 3. GENERATE 2D POWER SPECTRUM, FILTER WITH E.G. 3X3 BLURRNG KERNEL STEPANGLES FROMO DEGREES THROUGH 90 (1/2DEG) GENERATE NORMALIZED VECTOR, WITH POWER VALUE AS NUMERATOR, AND MOVING AVERAGED POWER VALUE AS DENOMINATOR 6. INTEGRATE VALUES AS SOME THRESHOLD, GIVING A SINGLE INTEGRATED VALUE FOR THIS ANGLE 7. END STEP ON ANGLES 8 FIND TOP ONE OR TWO OR THREE "PEAKS" FROM THE ANGLES IN LOOP 4, THEN FOR EACH PEAK. 9. STEP SCALE FROM 25% TO 400%, STEP - 1.01 10 ADD THE NORMAZED POWER VALUES CORRESPONDING TO THE 'N' SCALED FREOUENCES OF STANDARD KEEP TRACK OF HOHEST VALUE IN LOOP 12. END LOOP 9 AND 8, DETERMINE HIGHEST VALUE 3. ROTATION AND SCALE NOW FOUND 4. PERFORM TRADITIONAL MATCHED FILTER TO FIND EXACT SPATAL OFFSET 5. PERFORMANY "FNE UNING." TO PRECISELY DETERMINE ROTATION, SCALE, OFFSET U.S. Patent Apr. 7, 2009 Sheet 31 of 51 US 7,515,733 B2

9.20€

NO||LOES-B

22O2

9|02 C]/\/ HOLBEANOO \//O HOLHEANOO

|k||0:0 U.S. Patent Apr. 7, 2009 Sheet 32 Of 51 US 7,515,733 B2

990€.

N?VS) TlOH_LNOO U.S. Patent Apr. 7, 2009 Sheet 33 Of 51 US 7,515,733 B2

Z| HEC]OOEC]

BELLVWBO

U.S. Patent Apr. 7, 2009 Sheet 34 of 51 US 7,515,733 B2

1222

OOOOOOOH G1206 J.O.O.O.O.D.. O O:O:O:O:O:O:O: Of

123, 1208B5 s

(SIGNATURE BLOCK) 1208Ba/ 1208Aas 1208

FG 42

12O6 (SUB-BLOCK) U.S. Patent Apr. 7, 2009 Sheet 35. Of 51 US 7,515,733 B2

96eu||ueeleeuO U.S. Patent Apr. 7, 2009 Sheet 36 of 51 US 7,515,733 B2

?till Sex 32n ES5ill S2

Welcome to MarcCentre Offer Digimarc's Locator Service MarcCentre is an on-line service which links your up-to-date contact information to every image you watermark. You provide the details you want consumers of your image to see. Each tine someone reads one of your water marked images, they can access this information about you and your work with a single mouse click. Subscribe now to give each of your images a persistent identity, F communicating your copyright, and bringing consumers directly to G 44 you. Take advantage of the special offers available to you now, for a limited time. To find out more about how MarcCentre works and how you can get started taking advantage of this locator service, browse the MarcCentre overview page. If you are already a subscriber, you can go to Member Services to view or update your contact details. To look up image creator/distributor information, you can search by Creator ID or by other, more general, criteria. For the technically minded, find out what's going on behind the SCeneS.

oetased":Phot 4.O"Isree ores 2S9.gimarcage. ge. I state feedeck 2 are DiginarC Copyright Notice About the Site U.S. Patent Apr. 7, 2009 Sheet 37 Of 51 US 7,515,733 B2

<2- r

Company ...... '...'...... ----...... "

Address ...... ------Cit y s ::::::::::::::::::::::::::::::::::::::::: F G 45 State/Province Zip/Postal Code Country Phone Number Fax Number Email Address Your Home Page http:// You can control how much of your contact information is displayed when a potential customer finds your profile on MarcCentre. Use the selections below to tell us what you want people to see. Phone Number N1ailing Address Email Account s: Show :: Show :: Show ; : Don't show ; : Don't show i: Don't show Yes, I want to be on your mailing lists X Please select your profession (one only). s: Photographer 3: Illustrator :: Other In order to communicate your primary area of focus or specialty to potential customers, please select one of the options from the list below - or- type in your own in the space next to "Custom Specialty" Photographer Illustrator U.S. Patent Apr. 7, 2009 Sheet 38 of 51 US 7,515,733 B2

https://www.digimarc.com/owa devowa/REGINS. startup Digimarc displays an Image of the Day showcase on Our web site. Would you like to participate? :: Yes. :: Not at this time. For security purposes, you will need to setup a password that allows you to edit your profile information. This password along with your creator D are required before you can update your contact information. Password Re-enter to verify

The standard fee for the MarcCentre service is $150 (US). From now until December 31st, 1996 you can subscribe for only $79 (US), over 50% off the regular price For security reasons, the following Credit card information is captured using secured sockets technology. Credit Card ; : Visa :: MasterCard ; : American Express Card Number Name Shown on Card Expiration

When you are satisfied with your entries on this form press the "Part 1 Complete" button and continue with the second part of the subscription sign-up. If you wish to cancel, press the Back button on your browser. Partran Completecomplete FIG 46 Diginarc Copyright Notice U.S. Patent Apr. 7, 2009 Sheet 39 Of 51 US 7,515,733 B2

F.G. 47 Fe)Adobe Photoshop File Edit image Layer Select 5É View Window Help

53233;áé: Last filter Ctrl+F Žix Fade. Shft--Ctrl+F al Artistic Blu Brush Strokes Distort Noise , Pixelate 8. & Render T Sharpen I Sketch Stylize &Q, Texture % y Sésé 25: I d W

loog oc. 591 K591 K Draw selection from center or subtract from existing selection. U.S. Patent Apr. 7, 2009 Sheet 40 of 51 US 7,515,733 B2

F.G. 48 eAdobe Photoshop Eile Edit image layer Select Filter View Window Help 2.É.33% o

225se7

ZiSSS

Copyright info Creator O PictureMarc etno

Type of Use: C. Restricted

O Royalty Free

D Adult Content

Water mark Durability

less visible nore visible

less durable nore durable

d draw reclangular selection or now effort selection outfire. Usa Shii, A. and Ctrl of adosional options, U.S. Patent Apr. 7, 2009 Sheet 41 of 51 US 7,515,733 B2

FG. 49 eAdobe Photoshop File Edit Image Layer Select Filter View Window Help

33%33%

S&SSPN as NN Éx 222222223,52333

Click the Register button to obtain your own Creator D via the Internet or contact Digimarc Corporation at

http://www.diginarc.corn/register U.S. +1-800-664-3277 int. --SO3-223-O27

book Doc. 59 KV59K Draw rectangular selection or move fort, selection outline. Use Shilt. Aft, and C for additional options. U.S. Patent Apr. 7, 2009 Sheet 42 of 51 US 7,515,733 B2

33%3.3%

Doc. 591K591 K draw rectangular selection of Thove from selection outlire. Use Shift, Al, and Ctrl for additional options U.S. Patent Apr. 7, 2009 Sheet 43 of 51 US 7,515,733 B2

FIG 51 eAdobe Photoshop Fi le Edit image layer Select 56 View Window Help 2 353233%: Embed Watermark Ctrl+F %Ox Fade. Shift--Ct--F - Artistic Blur

Brush Strokes Distort Noise Pixelate Render Sharpen Sketch Stylize sa Texture y 97 Video Other

o Dox: 591 K591K Draw rectangular selection of move from selection outline. Use Shift. Ali, and Ctrl for additional options.

U.S. Patent US 7,515,733 B2

U.S. Patent Apr. 7, 2009 Sheet 46 of 51 US 7,515,733 B2

32 --- CY NS1 MarcCentre 1 MarcCentre Image Creator Search Enter the Inage Creator ID in the box below and press "Submit Search". If it is a valid ID, the contact information details will be listed. Image Creator ID: Submit Search If you want to search for specific image creators by a variety of criteria such as last name, specially, city and/or state then press the "General Search" button below. FG 54

DiginarC Copyright Notice U.S. Patent Apr. 7, 2009 Sheet 47 of 51 US 7,515,733 B2

eAdobe Photoshop File Edit image Layer Select Filter View Window Help

ÉE332%x

Copyright info

Creator D: OOOOl

More info

Organization D. 4S5

Oltem ID oransaction ID 234 w

Type of Use O Restricted O Royalty Free

Adult Content

Watermark Rurability Bump Size E. a

less visible more visible

less durable ?nore durable

Oraw rectangulat selection of move from selection cune. Use Shit. Ali, and Ctrl to additional options. U.S. Patent Apr. 7, 2009 Sheet 48 of 51 US 7,515,733 B2

eAdobe Photoshop File Edit image layer Select Filer View Window Help 2 Zon 27 (key (SXR5-se, 3éá53%22x Creator O: OOOO Organization ID: 456

Transaction ID: 234 Type of Use: Restricted

( - W k 9. For information about the creator of this image click ebook tip the Web Lookup button or contact the Digimarc V Locator Service.

http://www.diginarc.com/cgi-bin/cipl?-OOOO1 + 456+ Fax Back ---800-223-O 27

dot 59 K594 2222.2% 3:252 draw tectangular selection of nove selection outine use Shit. Ait. and C for additional optionsies U.S. Patent Apr. 7, 2009 Sheet 49 of 51 US 7,515,733 B2

GI)ETTOE«C»

U.S. Patent Apr. 7, 2009 Sheet 50 Of 51 US 7,515,733 B2

U.S. Patent US 7,515,733 B2

US 7,515,733 B2 1. 2 METHODS AND ARRANGEMENTS stored pseudo random sequence which is XORed with the EMPLOYING DIGITAL CONTENT ITEMS digital data and combined with the video. European application EP 581,317 discloses a system for RELATED APPLICATION DATA redundantly marking images with multi-bit identification codes. Each “1” (“O) bit of the code is manifested as a slight This application is a continuation of application Ser. No. increase (decrease) in values around a plurality of 10/821,414, filed Apr. 9, 2004 (now U.S. Pat. No. 7,035,427), spaced apart “signature points.” Decoding proceeds by com which is a continuation of application Ser. No. 09/545,174. puting a difference between a suspect image and the original, filed Apr. 6, 2000 (now U.S. Pat. No. 6,775,392), which is a unencoded image, and checking for pixel perturbations continuation of application Ser. No. 08/746,613, filed Nov. 10 around the signature points. 12, 1996 (now U.S. Pat. No. 6,122,403). PCT application WO95/14289 describes the present appli cant’s prior work in this field. MICROFICHEAPPENDIX Komatsu et al., describe an image marking technique in their paper “A Proposal on Digital Watermark in Document A microfiche appendix, containing 1077 pages on thirteen 15 Image Communication and Its Application to Realizing a microfiche, was submitted in application Ser. No. 08/746, Signature.” Electronics and Communications in Japan, Part 613, now U.S. Pat. No. 6,122.403. The complete disclosure of 1, Vol. 73, No. 5, 1990, pp. 22-23. The work is somewhat that application and patent is incorporated herein by refer difficult to follow but apparently results in a simple yes/no CCC. determination of whether the watermark is present in a Sus pect image (e.g. a 1 bit encoded message). TECHNICAL FIELD There is a large body of work regarding the embedding of digital information into video signals. Many perform the The present technology relates to identification of content embedding in the non-visual portion of the signal Such as in items, including , audio files and video files, the vertical and horizontal blanking intervals, but others together with arrangements and methods related to Such iden 25 embed the information “in-band'.(i.e. in the visible video tification. signal itself). Examples include U.S. Pat. Nos. 4,528,588; 4.595,950, and 5.319,453: European application 441,702: BACKGROUND and Matsui et al., “Video-Steganography: How to Secretly Embed a Signature in a Picture.” IMA Intellectual Property Hiding data in imagery or audio is a technique well known 30 Project Proceedings, January 1994, Vol. 1, Issue 1, pp. 187 to artisans in the field, and is termed “steganography.” There 205. are a number of diverse approaches to, and applications of There are various consortium research efforts underway in Steganography. A brief Survey follows: Europe on copyright marking of video and multimedia. A British patent publication 2,196,167 to Thorn EMI dis Survey of techniques is found in ''Access Control and Copy closes a system in which an audio recording is electronically 35 right Protection for Images (ACCOPI), WorkPackage 8: mixed with a marking signal indicative of the owner of the Watermarking.” Jun. 30, 1995, 46 pages. A new project, recording, where the combination is perceptually identical to termed TALISMAN, appears to extend certain of the the original. U.S. Pat. Nos. 4.963,998 and 5,079,648 disclose ACCOPI work. Zhao and Koch, researchers active in these variants of this system. projects, provide a Web-based electronic media marking Ser 40 Vice known as Syscop. U.S. Pat. No. 5,319,735 to Bolt, Berenak & Newman rests Aura reviews many issues of Steganography in his paper on the same principles as the earlier Thorn EMI publication, “Invisible Communication.” Helskinki University of Tech but additionally addresses psycho-acoustic masking issues. nology, Digital Systems Laboratory, Nov. 5, 1995. U.S. Pat. Nos. 4.425,642; 4.425,661; 5,404,377 and 5,473, Sandford II, et al. review the operation of their May, 1994, 631 to Moses disclose various systems for imperceptibly 45 image steganography program (BMPEMBED) in “The Data embedding data into audio signals—the latter two patents Embedding Method.” SPIE Vol. 2615, Oct. 23, 1995, pp. particularly focusing on neural network implementations and 226-259. perceptual coding details. A British company, Highwater FBI, Ltd., has introduced a U.S. Pat. No. 4,943,973 to AT&T discloses a system software product which is said to imperceptibly embed iden employing spread spectrum techniques for adding a low level 50 tifying information into photographs and other graphical noise signal to other data to convey auxiliary data therewith. images. This technology is the Subject of European patent The patent is particularly illustrated in the context of trans applications 94.00971.9 (filed Jan. 19, 1994), 9504221.2 mitting network control signals along with digitized Voice (filed Mar. 2, 1995), and 9513790.7 (filed Jul. 3, 1995), the signals. first of which has been laid open as PCT publication WO U.S. Pat. No. 5,161,210 to U.S. Philips discloses a system 55 95/20291. in which additional low-level quantization levels are defined Walter Bender at M.I.T. has done a variety of work in the on an audio signal to convey, e.g., a copy inhibit code, there field, as illustrated by his paper “Techniques for Data Hid with. ing.” Massachusetts Institute of Technology, Media Labora U.S. Pat. No. 4,972,471 to Gross discloses a system tory, January 1995. intended to assist in the automated monitoring of audio (e.g. 60 Dice, Inc. of Palo Alto has developed an audio marking radio) signals for copyrighted materials by reference to iden technology marketed under the name Argent. While a U.S. tification signals subliminally embedded therein. Patent Application is understood to be pending, it has not yet U.S. Pat. No. 5.243,423 to DeJean discloses a video stega been issued. nography system which encodes digital data (e.g. program Tirkeletal, at Monash University, have published a variety syndication verification, copyright marking, media research, 65 of papers on “electronic watermarking including, e.g., closed captioning, or like data) onto randomly selected video “Electronic Water Mark.” DICTA-93, Macquarie University, lines. DeJean relies on television sync pulses to trigger a Sydney, Australia, December, 1993, pp. 666-673, and “A US 7,515,733 B2 3 4 Digital Watermark. IEEE International Conference on Further features of the invention will become apparent with Image Processing, Nov. 13-16, 1994, pp. 86-90. reference to the following detailed description and accompa Coxetal, of the NEC Technical Research Institute, discuss nying drawings. various data embedding techniques in their published NEC technical report entitled “Secure Spread Spectrum Water BRIEF DESCRIPTION OF THE DRAWINGS marking for Multimedia. December, 1995. Möller et al. discuss an experimental system for impercep FIG. 1 is a simple and classic depiction of a one dimen tibly embedding auxiliary data on an ISDN circuit in “Rech sional digital signal which is discretized in both axes. nergestutzte Steganographie: Wie Sie Funktioniert und FIG. 2 is a general overview, with detailed description of warum folglich jede Reglementierung Von Verschlusselung 10 steps, of the process of embedding an “imperceptible” iden unsinnig ist.” DuD, Datenschutz und Datensicherung, 18/6 tification signal onto another signal. (1994) 318-326. The system randomly picks ISDN signal FIG.3 is step-wise description of how a suspected copy of samples to modify, and Suspends the auxiliary data transmis an original is identified. sion for signal samples which fall below a threshold. FIG. 4 is a schematic view of an apparatus for pre-exposing There are a variety of shareware programs available on the 15 film with identification information. internet (e.g. "Stego” and “White Noise Storm’) which gen FIG. 5 is a diagram of a “black box' embodiment. erally operate by Swapping bits from a to-be-concealed mes FIG. 6 is a schematic block diagram of the embodiment of sage stream into the least significant bits of an image or audio FIG.S. signal. White Noise Storm effects a randomization of the data FIG. 7 shows a variant of the FIG. 6 embodiment adapted to enhance its concealment. to encode successive sets of input data with different code words but with the same noise data. SUMMARY FIG. 8 shows a variant of the FIG. 6 embodiment adapted to encode each frame of a videotaped production with a The invention provides methods for decoding messages unique code number. from composite signals formed by embedding a message into 25 FIGS. 9A-9C are representations of an industry standard a host signal. These messages are encoded such that they are noise second. Substantially imperceptible in the composite signal. The FIG. 10 shows an integrated circuit used in detecting stan invention applies to a variety of different types of signals, dard noise codes. including images, audio and video. FIG.11 shows a process flow for detecting a standard noise One aspect of the invention is a method of decoding a 30 code that can be used in the FIG. 10 embodiment. message embedded into a code signal, which is embedded FIG. 12 is an embodiment employing a plurality of detec into a host signal. Together, the code and host signals form a tOrS. composite signal. The message is comprised of one or more FIG. 13 shows an embodiment in which a pseudo-random symbols, each being associated with a predetermined signal noise frame is generated from an image. characteristic. Before message decoding commences, the 35 FIG. 14 illustrates how statistics of a signal can be used in composite signal may be modified, either by unintentional or aid of decoding. intentional manipulation Such as compression, filtering, FIG. 15 shows how a signature signal can be preprocessed transmission, analog to digital and digital to analog conver to increase its robustness in view of anticipated distortion, e.g. Sion, etc. As such, the decoding process operates on a poten MPEG. tially modified version of the composite signal. To read the 40 FIGS. 16 and 17 show embodiments in which information message, the decoding process correlates predetermined sig about a file is detailed both in a header, and in the file itself. nal characteristics with the potentially modified composite FIGS. 18-20 show details relating to embodiments using signal to decode message symbols. rotationally symmetric patterns. Another aspect of the invention is a method of decoding a FIGS. 21A and 21B shoe encoding “bumps' rather than message Steganographically embedded into a host signal. 45 Prior to message decoding, a message is embedded into the pixels. host signal to form a composite signal in which the message FIGS. 22-26 detail aspects of a security card. is substantially imperceptible. The message embedder FIG. 27 is a diagram illustrating a network linking method using information embedded in data objects that have inher adjusts the host signal to encode a signal characteristic asso ent noise. ciated with message symbol. The decoding process analyzes 50 a potentially modified version of the composite signal to FIGS. 27A and 27B show a typical web page, and a step in gather evidence of the characteristic associated with the mes its encapsulation into a self extracting web page object. sage symbol. Based on the evidence, it determines whether FIG. 28 is a diagram of a photographic identification docu the message symbol is present in the potentially modified ment or security card. version of the composite signal. In contrast to the embodi 55 FIGS. 29 and 30 illustrate two embodiments by which ment described previously, this embodiment does not neces Subliminal digital graticules can be realized. sarily require the use of correlation to decode a message. FIG. 29.A shows a variation on the FIG. 29 embodiment. Yet another aspect of the invention is a method of decoding FIGS. 31A and 31B show the phase of spatial frequencies a message that uses one or more known message symbols to along two inclined axes. check the validity of a message extracted from a potentially 60 FIGS. 32A-32C show the phase of spatial frequencies modified version of a composite signal. The method analyzes along first, second and third concentric rings. the composite signal to gather evidence of characteristics FIGS. 33A-33E show steps in the registration process for a associated with the message symbols in the message. Based Subliminal graticule using inclined axes. on the evidence, it estimates message symbols. It compares FIGS. 34A-34E show steps in the registration process for a the estimated message symbols with one or more message 65 Subliminal graticule using concentric rings. symbols known to be included in the message to assess FIGS. 35A-35C shows further steps in the registration whether the composite signal has a valid message. process for a Subliminal graticule using inclined axes. US 7,515,733 B2 5 6 FIGS. 36A-36D show another registration process that depicted in FIG. 1 has the value of 2 to the fourth power, or “4 does not require a 2D FFT. bits.” giving 16 allowed states of the sample value. FIG. 37 is a flow chart summarizing a registration process For audio information Such as Sound waves, it is commonly for Subliminal graticules. accepted that the digitization process discretizes a continuous FIG.38 is a block diagram showing principal components phenomena both in the time domain and in the signal level of an exemplary wireless telephony system. domain. As such, the process of digitization itself introduces FIG. 39 is a block diagram of an exemplary Steganographic a fundamental error Source, in that it cannot record detail encoder that can be used in the telephone of the FIG. 38 smaller than the discretization interval in either domain. The system. industry has referred to this, among other ways, as “aliasing FIG. 40 is a block diagram of an exemplary Steganographic 10 in the time domain, and "quantization noise' in the signal decoder that can be used in the cell site of the FIG. 1 system. level domain. Thus, there will always be a basic error floor of FIGS. 41A and 41B show exemplary bit cells used in one a digital signal. Pure quantization noise, measured in a root form of encoding. mean square sense, is theoretically knownto have the value of FIG. 42 shows a hierarchical arrangement of signature one over the square root of twelve, or about 0.29 DN, where blocks, sub-blocks, and bit cells used in one embodiment. 15 DN stands for Digital Number or the finest unit increment of FIG. 43 is a general overview of a computer system linked the signal level. For example, a perfect 12-bit digitizer will by using information embedded in objects. have 4096 allowed DN with an innate root means square noise FIG. 44 is a representative computer screen generated in floor of -0.29 DN. accordance with the present invention. All known physical measurement processes add additional FIG. 45 is a representative computer screen generated in noise to the transformation of a continuous signal into the accordance with the present invention. digital form. The quantization noise typically adds in quadra FIG. 46 is a representative computer screen generated in ture (square root of the mean squares) to the “analog noise' of accordance with the present invention. the measurement process, as it is sometimes referred to. FIG. 47 is a representative computer screen generated in With almost all commercial and technical processes, the accordance with the present invention. 25 use of the scale is used as a measure of signal and FIG. 48 is a representative computer screen generated in noise in a given recording medium. The expression “signal accordance with the present invention. to-noise ratio” is generally used, as it will be in this disclo FIG. 49 is a representative computer screen generated in Sure. As an example, this disclosure refers to signal to noise accordance with the present invention. ratios in terms of signal power and noise power, thus 20 dB FIG. 50 is a representative computer screen generated in 30 represents a 10 times increase in signal amplitude. accordance with the present invention. In summary, this embodiment embeds an N-bit value onto FIG. 51 is a representative computer screen generated in an entire signal through the addition of a very low amplitude accordance with the present invention. encodation signal which has the look of pure noise. N is FIG. 52 is a representative computer screen generated in usually at least 8 and is capped on the higher end by ultimate accordance with the present invention. 35 signal-to-noise considerations and “bit error in retrieving FIG. 53 is a representative computer screen generated in and decoding the N-bit value. As a practical matter, N is accordance with the present invention. chosen based on application specific considerations, such as FIG. 54 is a representative computer screen generated in the number of unique different “signatures” that are desired. accordance with the present invention. 40 To illustrate, if N=128, then the number of unique digital FIG.55 is a representative computer screen generated in signatures is in excess of 1038 (2128). This number is accordance with the present invention. believed to be more than adequate to both identify the mate FIG. 56 is a representative computer screen generated in rial with sufficient statistical certainty and to index exact sale accordance with the present invention. and distribution information. FIG. 57 is a representative computer screen generated in 45 The amplitude or power of this added signal is determined accordance with the present invention. by the aesthetic and informational considerations of each and FIG. 58 is a representative computer screen generated in every application using the present methodology. For accordance with the present invention. instance, non-professional video can stand to have a higher FIG. 59 is a representative computer screen generated in embedded signal level without becoming noticeable to the accordance with the present invention. 50 average human eye, while high precision audio may only be able to accept a relatively small signal levellest the human ear DETAILED DESCRIPTION perceive an objectionable increase in “hiss'. These state ments are generalities and each application has its own set of In the following discussion of an illustrative embodiment, criteria in choosing the signal level of the embedded identi the words “signal” and “image are used interchangeably to 55 fication signal. The higher the level of embedded signal, the refer to both one, two, and even beyond two dimensions of more corrupted a copy can be and still be identified. On the digital signal. Examples will routinely switchback and forth other hand, the higher the level of embedded signal, the more between a one dimensional audio-type digital signal and a objectionable the perceived noise might be, potentially two dimensional image-type digital signal. impacting the value of the distributed material. In order to fully describe the details of an illustrative 60 To illustrate the range of different applications to which embodiment, it is necessary first to describe the basic prop applicant's technology can be applied, the present specifica erties of a digital signal. FIG. 1 shows a classic representation tion details two different systems. The first (termed, for lack of a one dimensional digital signal. The X-axis defines the of a better name, a "batch encoding system), applies identi index numbers of sequence of digital 'samples, and the fication coding to an existing data signal. The second (termed, y-axis is the instantaneous value of the signal at that sample, 65 for lack of a better name, a “real time encoding system), being constrained to exist only at a finite number of levels applies identification coding to a signal as it is produced. defined as the “binary depth of a digital sample. The example Those skilled in the art will recognize that the principles of US 7,515,733 B2 7 8 applicant's technology can be applied in a number of other acceptable perceived noise amplitude, and the resultant com contexts in addition to these particularly described. posite signal added to the original to become the distributable The discussions of these two systems can be read in either signal. order. Some readers may find the latter more intuitive than the The original signal, the N-bit identification word, and all N former; for others the contrary may be true. individual embedded code signals are then stored away in a secured place. A suspect signal is then found. This signal may Batch Encoding have undergone multiple copies, compressions and decom The following discussion of a first class of embodiments is pressions, resamplings onto different spaced digital signals, best prefaced by a section defining relevant terms: The original signal refers to either the original digital sig transfers from digital to analog back to digital media, or any 10 combination of these items. IF the signal still appears similar nal or the high quality digitized copy of a non-digital original. to the original, i.e. its innate quality is not thoroughly The N-bit identification word refers to a unique identifica destroyed by all of these transformations and noise additions, tion binary value, typically having N range anywhere from 8 then depending on the signal to noise properties of the embed to 128, which is the identification code ultimately placed onto ded signal, the identification process should function to some the original signal via the disclosed transformation process. 15 objective degree of statistical confidence. The extent of cor In the illustrated embodiment, each N-bit identification ruption of the Suspect signal and the original acceptable per words begins with the sequence of values 01 01, which is ceived noise level are two key parameters in determining an used to determine an optimization of the signal-to-noise ratio expected confidence level of identification. in the identification procedure of a Suspect signal (see defi The identification process on the Suspected signal begins nition below). by resampling and aligning the Suspected signal onto the The mth bit value of the N-bit identification word is either digital format and extent of the original signal. Thus, if an a Zero or one corresponding to the value of the mth place, image has been reduced by a factor of two, it needs to be reading left to right, of the N-bit word. E.g., the first (m=1) bit digitally enlarged by that same factor. Likewise, if a piece of value of the N=8 identification word 01110100 is the value 0: music has been “cut out, but may still have the same sam the second bit value of this identification word is 1, etc. 25 pling rate as the original, it is necessary to register this cut-out The mth individual embedded code signal refers to a sig piece to the original, typically done by performing a local nal which has dimensions and extent precisely equal to the digital cross-correlation of the two signals (a common digital original signal (e.g. both area 512 by 512 digital image), and operation), finding at what delay value the correlation peaks, which is (in the illustrated embodiment) an independent then using this found delay value to register the cut piece to a pseudo-random sequence of digital values. “Pseudo' pays 30 segment of the original. homage to the difficulty in philosophically defining pure ran Once the Suspect signal has been sample-spacing matched domness, and also indicates that there are various acceptable and registered to the original, the signal levels of the suspect ways of generating the “random' signal. There will be exactly signal should be matched in an rms sense to the signal level of N individual embedded code signals associated with any the original. This can be done via a search on the parameters given original signal. 35 ofoffset, amplification, and gamma being optimized by using The acceptable perceived noise level refers to an applica the minimum of the mean squared error between the two tion-specific determination of how much “extra noise, i.e. signals as a function of the three parameters. We can call the amplitude of the composite embedded code signal described Suspect signal normalized and registered at this point, or just next, can be added to the original signal and still have an normalized for convenience. acceptable signal to sell or otherwise distribute. This disclo 40 The newly matched pair then has the original signal Sub Sure uses a 1 dB increase in noise as a typical value which tracted from the normalized suspect signal to produce a dif might be acceptable, but this is quite arbitrary. ference signal. The difference signal is then cross-correlated The composite embedded code signal refers to the signal with each of the N individual embedded codesignals and the which has dimensions and extent precisely equal to the origi peak cross-correlation value recorded. The first four bit code nal signal, (e.g. both are a 512 by 512 digital image), and 45 (0.101) is used as a calibrator both on the mean values of the which contains the addition and appropriate attenuation of the Zero value and the one value, and on further registration of the N individual embedded code signals. The individual embed two signals if a finer signal to noise ratio is desired (i.e., the ded signals are generated on an arbitrary scale, whereas the optimal separation of the 01.01 signal will indicate an optimal amplitude of the composite signal must not exceed the pre-set registration of the two signals and will also indicate the prob acceptable perceived noise level, hence the need for “attenu 50 able existence of the N-bit identification signal being ation of the Nadded individual code signals. present.) The distributable signal refers to the nearly similar copy of The resulting peak cross-correlation values will form a the original signal, consisting of the original signal plus the noisy series of floating point numbers which can be trans composite embedded code signal. This is the signal which is formed into 0’s and 1s by their proximity to the mean values distributed to the outside community, having only slightly 55 of 0 and 1 found by the 01 01 calibration sequence. If the higher but acceptable “noise properties” than the original. Suspect signal has indeed been derived from the original, the A suspect signal refers to a signal which has the general identification number resulting from the above process will appearance of the original and distributed signal and whose match the N-bit identification word of the original, bearing in potential identification match to the original is being ques mind either predicted or unknown “bit error” statistics. Sig tioned. The Suspect signal is then analyzed to see if it matches 60 nal-to-noise considerations will determine if there will be the N-bit identification word. some kind of “bit error in the identification process, leading The detailed methodology of this first embodiment begins to a form of X% probability of identification where X might by stating that the N-bit identification word is encoded onto be desired to be 99.9% or whatever. If the suspect copy is the original signal by having each of them bit values multiply indeed not a copy of the original, an essentially random their corresponding individual embedded code signals, the 65 sequence of 0s and 1’s will be produced, as well as an resultant being accumulated in the composite signal, the fully apparent lack of separation of the resultant values. This is to Summed composite signal then being attenuated down to the say, if the resultant values are plotted on a histogram, the US 7,515,733 B2 10 existence of the N-bit identification signal will exhibit strong The total bits required now are at 4 bits for the 01.01 calibra bi-level characteristics, whereas the non-existence of the tion sequence, 16 for the main identification, 8 for the version, code, or the existence of a different code of a different origi and we now throw in another 4 as a further error checking nal, will exhibit a type of random gaussian-like distribution. value on the first 28 bits, giving 32 bits as N. The final 4 bits This histogram separation alone should be sufficient for an can use one of many industry standard error checking meth identification, but it is even stronger proof of identification ods to choose its four values. when an exact binary sequence can be objectively repro We now randomly determine the 16 bit main identification duced. number, finding for example, 1101 0001 1001 1110; our first Specific Example versions of the original sold will have all O’s as the version 10 identifier, and the error checking bits will fall out where they Imagine that we have taken a valuable picture of two heads may. We now have our unique 32 bit identification word of state at a cocktail party, pictures which are Sure to earn which we will embed on the original digital image. some reasonable fee in the commercial market. We desire to To do this, we generate 32 independent random 4000 by sell this picture and ensure that it is not used in an unautho 4000 encoding images for each bit of our 32 bit identification rized or uncompensated manner. This and the following steps 15 word. The manner of generating these random images is are summarized in FIG. 2. revealing. There are numerous ways to generate these. By far Assume the picture is transformed into a positive the simplest is to turn up the gain on the same Scanner that was print. We first scan this into a digitized form via a normal high used to scan in the original photograph, only this time placing quality scanner with a typical photometric a pure black image as the input, then Scanning this 32 times. spectral response curve. (It is possible to get better ultimate The only drawback to this technique is that it does require a signal to noise ratios by scanning in each of the three primary large amount of memory and that “fixed pattern” noise will be of the color image, but this nuance is not central to part of each independent “noise image.” But, the fixed pattern describing the basic process). noise can be removed via normal “dark frame' subtraction Let us assume that the Scanned image now becomes a 4000 techniques. Assume that we set the absolute black average by 4000 pixel monochrome digital image with a grey scale 25 value at digital number 100, and that rather than finding a 2 accuracy defined by 12-bit grey values or 4096 allowed lev DN rms noise as we did in the normal gain setting, we now els. We will call this the “original digital image' realizing that find an rms noise of 10 DN about each and every pixel’s mean this is the same as our “original signal” in the above defini value. tions. We next apply a mid-spatial-frequency bandpass filter During the Scanning process we have arbitrarily set abso 30 (spatial convolution) to each and every independent random lute black to correspond to digital value 30'. We estimate that image, essentially removing the very high and the very low there is a basic 2 Digital Number root mean square noise spatial frequencies from them. We remove the very low fre existing on the original digital image, plus a theoretical noise quencies because simple real-world error sources like geo (known in the industry as “shot noise') of the square root of metrical warping, splotches on Scanners, mis-registrations, the brightness value of any given pixel. In formula, we have: 35 and the like will exhibit themselves most at lower frequencies also, and so we want to concentrate our identification signal at brackets around the RMS noise merely out higher frequencies anyway, so there is no point in placing indicates that this is an expected average value, where it is too much identification signal into these frequencies if they clear that each and every pixel will have a random error will be the ones most prone to being attenuated. Therefore, individually. Thus, for a pixel value having 1200 as a digital our new filtered independent noise images will be dominated number or “brightness value', we find that its expected rms 45 by mid-spatial frequencies. On a practical note, since we are noise value is sqrt(1204)=34.70, which is quite close to 34.64, using 12-bit values on our scanner and we have removed the the square root of 1200. DC value effectively and our new rms noise will be slightly We furthermore realize that the square root of the innate less than 10 digital numbers, it is useful to boil this down to a brightness value of a pixel is not precisely what the eye 6-bit value ranging from -32 through 0 to 31 as the resultant perceives as a minimum objectionable noise, thus we come up 50 random image. with the formula: Next we add all of the random images together which have a 1 in their corresponding bit value of the 32-bit identifica