N-Tuple Method Robert M. Haralick Computer Science, Graduate Center City University of New York Outline N-Tuple Method Developed For Printed Character Recognition (1959) Each character is contained in an image of M × N pixels Each pixel is a binary 1 or a binary 0 Designed for table lookup hardware N-Tuple Method L randomly chosen pixel positions N-Tuple Method A small number of pixel positions are randomly selected Have multiple sets of such randomly selected pixel positions Each of the pixel positions has been thresholded and contains a binary 0 or a binary 1 Concatenate all the binary values to form a binary number Use this number to access an address in a memory array For each character class Have as many memory arrays as there are different randomly selected position sets N-Tuple Method M pattern sets of L randomly selected pixel positions A printed character produces M L-digit binary numbers b1;:::; bM K character classes Tmk lookup table for pattern set m and class k Tmk (bm) holds the fraction of times a character in the th training set of class k has the binary number bm for the m pattern set Compute M fk = minm=1 Tmk (bm) PM fk = m=1 Tmk (bm) Assign the character to unique class ck , if there is one, for which fk > 0 is highest Otherwise reserve decision N-Tuple Method Recast M pattern sets of L randomly selected pixel positions A printed character produces M binary numbers b1;:::; bM K character classes Tm lookup table for pattern set m Tm(bm) holds the set of classes associated with the binary th number bm for the m pattern set Compute \M f = m=1Tm(bm) Assign the character to unique class ck , if there is one, where ck 2 f and jf j = 1 Otherwise reserve decision Relations Each of the possible pixel positions is a variable Let X1;:::; XN be the N variables Let Ln be the possible values variable Xn can take Let R be the training set for one class N R ⊆ Ln n=1 Index Sets Definition I = fi1;:::; iK g is an index set if and only if for a given partial ordering ≤ on I, i1 < i2 < ··· < iK . Definition If I = fi1;:::; iK g is an index set, we define the Cartesian product K Li = Lik = Li1 × Li2 × · · · × LiK i2I k=1 We adopt a convention that if the index set I = ;, then i2I Li = ;. N-ary Relation Definition If I is an index set with jIj = N and R ⊆ i2I Li , then we say (I; R) is an Indexed N-ary Relation on the range sets indexed by I. We also say that (I; R) has dimension N. We can perform the usual set operations of union and intersection with the structures (I; R) and (J; S) and when I = J: (I; R) [ (I; S) = (I; R [ S) (I; R) \ (I; S) = (I; R \ S) Subset The subset relation also has the usual meaning. If R ⊆ i2I Li and S ⊆ i2I Li , then R ⊆ S if and only if (I; R) ⊆ (I; S) Index Function Definition Let J and M be index sets with J = fj1;:::; jjJjg M = fm1;:::; mjMjg J ⊂ M The index function fJM : J ! [jMj] is defined by fJM (j) = k where mk = j fJM operates on an index j from the smaller index set and specifies where – place k – in the larger index set that the index j can be found; thus mk = j. Index Function Example J = f2; 5; 8g M = f2; 4; 5; 6; 7; 8; 9g = fm1; m2; m3; m4; m5; m6; m7g Index function fJM fJM operates on an index j from the smaller index set and specifies where – place k – in the larger index set that the index j can be found; thus mk = j. j fJM (j) 2 1 5 3 8 6 Projection Definition Let I and J be index sets with I ⊂ J. The projection operator projecting a relation on the range sets indexed by J onto the range sets indexed by I is defined by πI (J; R) = (I; S) where S = f(x1;:::; xjIj) 2 Li j 9(a1;:::; ajJj) 2 R; afIJ (i) = xi ; i 2 Ig i2I We overload the definition of the projection operator so that it can operate on tuples as well as sets. πI(J; (a1;:::; ajJj)) = (I; (x1;:::; xjIj); where (x1;:::; xjIj) = (afIJ (1);:::; afIJ (jIj)) = (I; (afIJ (1);:::; afIJ (jIj))) Projection X2 πf2g(f1; 2g; R) R X1 πf1g(f1; 2g; R) Projections Proposition Let I and J be an index sets with I ⊆ J. If (J; A) ⊆ (J; B), then πI(J; A) ⊆ πI(J; B). Projections of Unions Proposition Let A and B be relations on the range sets indexed by index set J. Let I ⊂ J. Then πI(J; A [ B) = πI(J; A) [ πI(J; B) Projections of Intersections Proposition Let A and B be relations on the range sets indexed by index set J. Let I ⊂ J. Then πI(J; A \ B) ⊆ πI(J; A) \ πI(J; B) The following example shows that this proposition cannot be made stronger. J = f1; 2g,I = f1g A = f(a; α); (b; β)g, B = f(a; γ); (b; δ)g Then A \ B = ; so that πI(J; A \ B) = ;. But πI(J; A) = fa; bg and πI(J; B) = fa; bg so that πI(J; A) \ πI(J; B) = fa; bg. Inverse Projection X2 −1 f g \ −1 f g T2 πf1;2g( 1 ; T1) πf1;2g( 2 ; T2) X1 T1 f g × −1 f g \ −1 f g ( 1; 2 ; T1 T2) = πf1;2g( 1 ; T1) πf1;2g( 2 ; T2) Inverse Projection Definition Let I and J be index sets with I ⊂ J R ⊆ i2I Li The Inverse Projection of (I; R) with respect to index set J is defined by −1 πIJ (I; R) = f(J; (a1;:::; ajJj)) 2 (J; Lj ) j πI (J; (a1;:::; ajJj)) 2 (I; R)g j2J Inverse Projections Proposition Let I and J be index sets with I ⊆ J. Suppose (I; R) ⊆ (I; S). Then, −1 −1 πIJ (I; R) ⊆ πIJ (I; S) Projection and Inverse Projections Proposition Let I; J be index sets with I ⊆ J. Then −1 πI(πIJ (I; R)) = (I; R) −1 πIJ (πI(J; R)) ⊇ (J; R) Projection X2 πf2g(f1; 2g; R) R X1 πf1g(f1; 2g; R) Inverse Projection X2 −1 f g \ −1 f g T2 πf1;2g( 1 ; T1) πf1;2g( 2 ; T2) X1 T1 f g × −1 f g \ −1 f g ( 1; 2 ; T1 T2) = πf1;2g( 1 ; T1) πf1;2g( 2 ; T2) −1 f g f g πf1gπf12g( 1 ; T1) = ( 1 ; T1) Projections and Inverse Projections Corollary Let I and J be index sets with I ⊆ J. Suppose πI(J; S) = (I; R). −1 ⊇ Then πJI (I; R) (J; S). Corollary Let J1;:::; JK ; J be index sets with J1;:::; JK ⊆ J. Then (J; R) ⊆ π−1 π (J; R) \ π−1 π (J; R) \···\ π−1 π (J; R) J1J J1 J2J J2 JK J JK Projections of a Composite Relation Proposition Let I; J; K be index sets with I; J ⊆ K . Suppose −1 −1 (K ; R) = πIK (I; S) \ πJK (J; T ) Then πI(K ; R) ⊆ (I; S) πJ (K ; R) ⊆ (J; T ) Relation Join Definition Index sets: J1; J2 Index set: K = J1 [ J2 ⊆ Relation: R1 j2J1 Lj ⊆ Relation: R2 j2J2 Lj Relation: S ⊆ k2K Lk (K ; S) is the Relation Join of (J1; R1) and (J2; R2) if and only if (K ; S) = π−1 (J ; R ) \ π−1 (J ; R ) J1K 1 1 J2K 2 2 We write: (K ; S) = (J1; R1) ⊗ (J2; R2) Join Characterization Theorem Theorem Let I; J; K be index sets with K = I [ J. Let R ⊂ i2I Li and S ⊂ j2J Lj Then (I; R) ⊗ (J; S) = (K ; T ) where T = f(t ;:::; t ) 2 L j π (K ; (t ;:::; t ))g 2 (I; R) and 1 jK j k I 1 jK j k2K πJ (K ; (t1;:::; tjK j)) 2 (J; S)g Relation Join Example R S 1,3,5,6 2,3,4,5 1 (a,b,e,d) 1 (e,e,a,d) 2 (b,d,e,a) 2 (d,c,b,a) 3 (e,c,a,b) 3 (a,d,b,e) 4 (c,e,d,a) 4 (b,b,c,e) 5 (e,d,c,e) Then R ⊗ S is defined by R ⊗ S 1,2,3,4,5,6 (1,4) (a,b,b,c,e,d) (2,3) (b,a,d,b,e,a) (2,5) (b,e,d,c,e,a) (3,2) (e,d,c,b,a,b) (4,1) (c,e,e,a,d,a) Join Properties Proposition Let I; J; K ⊂ [N] be index sets. Let R ⊆ i2I Li S ⊆ j2J Lj T ⊆ k2K Lk Then, (I; R)⊗ (J; S) = (J; S) ⊗ (I; R) ((I; R) ⊗ (J; S)) ⊗ (K ; T ) = (I; R) ⊗ ((J; S) ⊗ (K ; T )) (I; R) ⊗ (I; R) = (I; R) (I; R) ⊆ (I; T ) implies (I; R) ⊗ (I; T ) = (I; R) N-Tuple Method [N] = f1;:::; Ng indexes for all pixel positions ([N]; Tc) training set for class c J1;:::; JM M sets of random subsets of [N], each having L elements πJm ([N]; Tc) = (Jm; Tmc); m = 1;:::; M analogous to tables N r 2 n=1 Ln is new measurement Assign r to unique class c, if there is one where πJm ([N]; r) 2 (Jm; Tmc); m = 1;:::; M Otherwise reserve decision N-Tuple Method Theorem The set of all tuples that could potentially be assigned to class c is f([N]; x) j π ([N]; x) 2 (J ; T ); m = 1;:::; Mg = ⊗M (J ; T ) Jm m mc m=1 m mc Acceptance Region Definition The Acceptance Region for a class is the set of all measurements that will be assigned to the class.