A Sheaf and Approach to Detecting Local Merging Relations in Digital Images

Chuan-Shen Hu Yu-Min Chung Department of Mathematics Department of Mathematics and Statistics National Taiwan Normal University University of North Carolina at Greensboro Taipei 116325, Taiwan Greensboro, North Carolina 27412, USA [email protected] y [email protected]

Abstract regions. The q-dimensional PH of filtration1 of topological spaces is a sequence of vector spaces that are connected by 2 This paper concerns a theoretical approach that com- linear transformations . The non-negative integer q denotes bines topological data analysis (TDA) and sheaf theory. the dimension of objects which are captured by the PH. For Topological data analysis, a rising field in mathematics and instance, the PH of q = 0 captures the changes of connected computer science, concerns the shape of the data and has components and q = 1 for holes in a filtration. been proven effective in many scientific disciplines. Sheaf Persistence barcodes (or simply barcodes) are a typical theory, a mathematics subject in algebraic geometry, pro- way to summarize the information of PH. A barcode of a vides a framework for describing the local consistency in q-dimensional object/generator in PH is a pair of numbers geometric objects. Persistent homology (PH) is one of the (b,d) where the object/generator is born at the value b and main driving forces in TDA, and the idea is to track changes dies at the value of d. One often refers to b, and d to the in geometric objects at different scales. The persistence di- birth and death, respectively, values. For instance, in Fig- agram (PD) summarizes the information of PH in the form ure 1, a 1-dimensional hole of the character “A” was born of a multi-. While PD provides useful information about at g2 and disappeared at g4, and hence the hole has the bar- the underlying objects, it lacks fine relations about the local code (2, 4). Barcodes in PH can be defined rigorously as consistency of specific pairs of generators in PD, such as the algebraic structure of PH [27, 21, 15, 25] (cf. Lemma the merging relation between two connected components in 2.2.1). The collection of all barcodes of q-dimensional ob- the PH. The sheaf structure provides a novel point of view jects is called the persistence diagram (PD) [27, 21, 15, 25]. for describing the merging relation of local objects in PH. It PD plays an essential role in the application and has widely is the goal of this paper to establish a theoretic framework applied and studied in computer vision [66, 9, 61], machine that utilizes the sheaf theory to uncover finer information learning [14, 1, 44, 53], /signal processing [67, 11], from the PH. We also show that the proposed theory can be medical science [50, 46, 22, 7, 10], and physics [62, 5]. applied to identify the merging relations of local objects in Although PD contains information about the changes of digital images. local objects, the reduction algorithm for obtaining Smith normal forms [26] of the connecting linear transformations would omit the merging relations of elements in PH. In other words, the PD is not enough for researchers who are 1. Introduction interested in the merging behaviors of certain local objects. For example, the top and bottom panels of Figure 2 (a) show Topological data analysis (TDA) is a branch of applied two different filtrations of binary images, but they share mathematics that aims to quantify topological characteris- the same persistence diagram P0 as shown in Figure 2 (b). tics, especially the q-dimensional Betti numbers denoted by The merging relation between connected components in the βq. For instance, β0, β1, and β2 represent the number of third image is different: one connects diagonally while the components, holes, or voids, respectively. Persistent ho- other connects horizontally. mology (PH), one of the main tools in TDA, tracks changes 1 of towered topological spaces induced by the original ob- A filtration of topological spaces is an inclusions X1 ⊆ X2 ⊆···⊆ Xn of topological spaces. jects [27, 21, 15, 25, 49, 6]. Figure 2 (a) shows two exam- 2In algebraic topology, one may also consider PH of modules and mod- ples of included spaces (also known as filtrations) of black ule homomorphisms. lular sheaf cohomologies on abstract simplicial complexes of real signals and their samplings. In the work, the con- cept of global and local sections on posets equipped with the Alexandrov topology [3, 4] is crucial for constructing the sheaf structures on digital/analog signals. Our work (a) g (b) g (c) g (d) g 1 2 3 4 is also motivated by the multi-parameter persistent homol-

4 4 ogy [20, 57, 35] and zigzag homology [16, 19, 18, 17]. The extension of barcodes to multi-parameter persistent ho- 3.5 3 mology is an active research area in TDA. As discussed in 3 [39, 40, 24], sheaf theory may be an appropriate tool to- 2 Death Value Death Value 2.5 wards that goals. The cellular sheaves considered in the pa- 1 2 1 2 3 4 2 2.5 3 3.5 4 per can be viewed as a form of zigzag homology and multi- Birth Value Birth Value parameter persistent homology. The proposed work aims

(e) P0 (f) P1 to capture local sections of certain local objects and their geometric meaning in digital images. Figure 1. (a) ∼ (d) is a filtration of black pixels of binary images. Our main contributions can be summarized in the fol- (c) Binary image “AI”, which has β0 =2 (connected components) lowing. and β1 = 1 (1-dimensional holes). (e) and (f) are persistence diagrams P0 = {(1, ∞), (3, 4)}, P1 = {(2, 4)} in dimension 0 • We propose the coincidence of pairs in the filtration of and 1 of the filtration respectively. binary images. The coincidence can be identified as global/local sections of a sheaf of the form (6). When q = 0, the coincidence in a short filtration can be used There are some works related to the behaviors of lo- to define local merging numbers as a heat of each cal objects in PH, such as Mapper [42, 59] and local binary image. (co)homology [30, 29]. In particular, Vandaele et al. [64] investigated the local Vietoris-Rips complexes of point • In the paper, we propose an approximation of local sec- clouds. By computing the branch numbers b and the first tions in a PH by its PD. The result allows practitioners Betti numbers β1 of the induced graph structures, the pairs to estimate the representatives of local objects in PH. (b,β1) correspond to the branch numbers and holes of lo- cal objects. The local pairs provide a heat map of branch The organization of the paper is as follows. We present our numbers and loop structures. Comparing to the global Betti main results in Section 2. The demonstration of generating numbers, it is additional geometric information for the ob- local merging numbers of binary images is shown in Sec- ject. The proposed work bases on a similar idea and pro- tion 3. The discussions, future works, and the conclusion vides a sheaf theoretical approach to describe the local be- are in Section 4. haviors of a geometric object. Comparison of these two methods will be discussed at the end of Section 2.3. 2. Cellular Sheaves Modeling Historically, sheaves were first developed as a tool for researching the nerve theory and fixed-point theorems [2, This section is separated into three parts. In Sec- 28, 13], while the recent trend is to study algebraic geome- tion 2.1, we briefly introduce the formal definition of cel- try [36]. Sheaf theory and algebraic geometry provide fruit- lular sheaves over posets. To define the local sections on B ful results and tools in analyzing local/global properties of cellular sheaves, the Alexandrov Topology and the -sheaf geometric objects, while the research of the combination of are also mentioned. In order to be self-contained, we pro- sheaves and TDA is still in its infancy. A sheaf F over a vide necessary notations and definitions. For further details X is a rule which assigns each open sub- on these topics, we refer readers to [24, 3, 4] and [47]. The- set U of X to an algebraic object F (U). Except objects, orem 2.2.2 in Section 2.2 approximates the local sections a sheaf F also assigns each pair V ⊆ U of open sets to a via barcodes. Finally, in Section 2.3, we apply our results to analyze local information of binary images. homomorphism ρUV : F (U) → F (V ) as an connection between two algebraic objects. The assignments of and U 2.1. Cellular Sheaves and Coincidence in PH V ⊆ U are analogous to the space of functions on U and restriction of functions on V . A poset (or ) (P, ≤) is a non-empty Recently, the combination of the sheaf theory and TDA set P equipped with a relation ≤ on P which satisfies the has been found its potential in modeling and analyzing real following properties: Whenever x,y,z ∈ P , (1) x ≤ x (2) data [31, 54, 56, 55, 8, 34, 24]. For example, Robinson [54] x ≤ y and y ≤ z implies x ≤ z, and (3) x ≤ y and y ≤ x proved the Nyquist sampling theorem by computing cel- implies x = y. 4

3

2 Death Value

1 1 2 3 4 Birth Value

(a) Two filtrations of black pixels of binary images (b) The persistence diagram P0

Figure 2. An example of two filtrations of black pixels that share the same persistence diagram.

a certain field [55, 23, 24]. By considering the Alexandrov topology [3, 4] A on the poset P generated by the sets Up’s of forms

Up = {q ∈ P : p ≤ q} (1) as an open basis B for the topology A, the assignment Up 7→ F (Up) := F (p) forms a B-sheaf structure on the topological space (P, A) [47]. The B-sheaf can be ex- tended as a sheaf of vector spaces over (P, A), which is the real meaning of “sheaves on topological spaces”—from the aspect of traditional sheaf theory [13] or algebraic geome- Figure 3. An example of inclusions separated from a filtration of try [36, 47]. binary images (cf. Figure 1 (a) ∼ (d)). These inclusion relations Because B = {U : p ∈ P } is an open basis for A, lead to the simplest cellular sheaf structure as in Equation (6)). p every open subset U in (P, A) can be expressed by U = F A F Sp∈U Up. To extend as a sheaf on (P, ), the (U) is Example 2.1.1. (1) Suppose X is a non-empty set and 2X defined as the vector space denotes its . Then the inclusion relation ⊆ on 2X X is a partial order on 2 . In this paper, we mainly consider F (U)= {(sp)p∈U : sq = ρp,q(sp) ∀ p ≤ q} , (2) ⊆ on 2S, where S is a subset of Z2. n (2) Let (x1,x2,...,xn), (y1,y2,...,yn) ∈ Z be n-tuples where every (sp)p∈U denotes an element in the Cartesian F over Z. Define (x1,x2,...,xn) ≤ (y1,y2,...,yn) if and only product Qp∈U (p) of vector spaces [32, 54, 56, 24]. El- n F if xi ≤ yi for all i ∈ {1, 2,...,n}. Then (Z , ≤) is a poset. ements in (U) are called local sections on U, which are the most important targets for observing the coincidence of A cellular sheaf over a poset (P, ≤) is a rule which as- elements from different F (p)’s. Here is an example: signs each element in P to a vector space over a fixed field F (e.g. Z2 or R) and preserves the ordering of elements in Example 2.1.2. Let P = {(0, 0), (0, 1), (1, 0), (1, 1)} be P via linear transformations. Here is the formal definition: a set of tuples in Z2. Then (P, ≤) is a poset, where ≤ is defined as in Example 2.1.1 (2). Any cellular sheaf F on P Definition ([55, 23]). Let (P, ≤) be a poset. A celluar can be represented as the commutative diagram sheaf of vector spaces (over a fixed field F) on P is a rule F which consists of the following data: ρ(0,1),(1,1) F (0, 1) / F (1, 1) (3) • For each p ∈ P , F associates a vector space F (p) O O

over F. ρ(0,0),(0,1) ρ(1,0),(1,1)

• For p ≤ q in P , there is an F-linear transformation ρ(0,0),(1,0) F (0, 0) / F (1, 0) ρp,q : F (p) → F (q) such that ρp,p is the identity map on F (p) and ρ ◦ ρ = ρ for every p ≤ q ≤ r. q,r p,q p,r of vector spaces and linear transformations.

In category theory, a cellular sheaf of vector spaces on a The local sections on U := U(0,1) ∪ U(1,0) = poset (P, ≤) is a from the category of elements and {(0, 1), (1, 0), (1, 1)} are pairs (s,t) in F (0, 1) × F (1, 0) partial relations in P to the category of vector spaces over such that ρ(0,1),(1,1)(s)= ρ(1,0),(1,1)(t) in F (1, 1). The concept of local sections described in Example Proof. By the same arguments as in Example 2.1.2, an el- 2.1.2 provides a bridge to investigate the relation between ement (si,sj) ∈ Hq(Xi) ⊕ Hq(Xj) is a local section of F (0, 1) and F (1, 0). A local section records elements s,t the cellular sheaf (6) if and only if ρi,k(si) = ρj,k(sj), as in F (0, 1), F (1, 0) respectively, which would be identified desired. as the same element in F (1, 1) via ρ•,•’s. More concrete examples for digital images and their sheaf structures are For example, in the filtration (a) ∼ (d) in Figure 1, the discussed in Section 2.3. component “I” has barcode (3, 4) since it was born at g3 The cellular sheaf can be viewed as a generalization and finally be merged into “A”. However, the persistence di- of persistent homology. For example, F in (3) is a 2- agram (Figure 1 (e)) only shows that “I” has barcode (3, 4) parameter persistent homology over the poset P . An one- while the merging information of “I” and “A” could not be parameter PH concluded in the diagram. As in Theorem 2.1.3, this local behavior between “A” ρ0,1 ρ1,2 Hq(X0) −−→ Hq(X1) −−→· · · −→ Hq(Xn) (4) and “I” is encoded as a local section of the cellular sheaf (7), by separating image g3 into two parts g3,1 = “A” and of topological spaces ∅ = X0 ⊆ X1 ⊆···⊆ Xn is a cellu- −1 −1 g3,2 = “I”. The inclusions g3,1(0) ֒→ g4 (0) and lar sheaf over P = {0, 1, 2,...,n} of homologies and linear −1 −1 g (0) ֒→ g (0) in Figure 3 lead the diagram 3 transformations ρi,i+1 : Hq(Xi) → Hq(Xi+1) induced by 3,2 4 the inclusions Xi ֒→ Xi+1 [33], where −1 ρ1 / −1 H0(g3,1(0)) H0(g4 (0)) (7) ρi,j := ρj−1,j ◦ · · · ◦ ρi,i+1 (5) O ρ2 and ρi,i are the identity maps on Hq(Xi) for 0 ≤ i

Hq(Xj) • ρi,j−1(si) ∈/ im(ρi−1,j−1) and ρi,j(si) ∈ im(ρi−1,j), 3 In the paper, we consider Hq(Xi)’s and ρi,i+1’s as vector spaces and where the death j may be ∞ if s is still alive at n. Z i linear transformations over the binary field 2. As Edelsbrunner and Harer mentioned in [26], the s has 4For finitely many homologies, we use the notation ⊕ to replace ×. i 5 To emphasis the merging relation between si and sj , here we ignore death j if it merges with an older class in Hq(Xj−1). The the linear transformation Hq(Xi) → Hq(Xj ) for i ≤ j. following approximation lemma can be viewed as a local version of barcodes of elements, which approximates the 2.3. Sheaf Structures on Binary Images upper bound of deaths for any s ,s who are coinciding at i j The subsection is separated into two parts. We first intro- some k. duce some notations and terminologies for binary images,

Lemma 2.2.1 (Approximation Lemma). Let ∅ = X0 ⊆ and then introduce the short filtrations for a single binary X1 ⊆···⊆ Xn be a filtration of topological spaces, q ≥ 0, image and apply its sheaves to compute the local merging 0 ≤ ij. Z . Typically, S = ([a,b]×[c,d])∩Z is a rectangle, called Suppose d>k, then i

Proof. We first assume there is an si ∈ Hq(Xi) such that natural cellular sheaf structure si and sj coincide at k in F. Because sj is born at j in F, ρ H (X ) 1 / H (X) (12) ρi,j(si) 6= sj. Hence sj is also born at 2 in G. In G, si q 1 q O and sj also coincide at 3, by the approximation lemma, the ρ2 death of sj in must ≤ 3, and it forces that sj has barcode in . (2, 3) G Hq(X2) Conversely, suppose sj ∈ Hq(Y2) has barcode (2, 3) in G G G G, then ρ2,3(sj) ∈ im(ρ1,3) where ρ•,•’s are the induced where ρ1, ρ2 are induced by the inclusion maps. Now we linear transformations from ρ•,• in F. This shows that have two short filtrations there is an si ∈ Hq(Y1) = Hq(Xi) such that ρi,k(si) = G G ρ (s ) = ρ (s ) = ρ (s ) i.e., s and s coincide at k G1 : ∅⊆ X1 ⊆ X1 ∪ X2 ⊆ X, 1,3 i 2,3 j j,k j i j (13) in F. G2 : ∅⊆ X2 ⊆ X1 ∪ X2 ⊆ X. and persistent homologies

ω1 γ 0 −→ Hq(X1) −→ Hq(X1 ∪ X2) −→ Hq(X), (14) ω2 γ 0 −→ Hq(X2) −→ Hq(X1 ∪ X2) −→ Hq(X), whenever q ≥ 0, where ω ,ω and γ are induced by the 1 2 (a) X1 (b) A1 (c) X1 \ A1 inclusion maps and γ ◦ ωi = ρi for i = 1, 2. 6 In addition, when clX (X1) and clX (X2) are disjoint , the canonical mapping

ω1 + ω2 : Hq(X1) ⊕ Hq(X2) → Hq(X1 ∪ X2) (15) defined by (ξ1,ξ2) 7−→ ξ1 + ξ2 is an isomorphism ([33] (d) X2 (e) A2 (f) X2 \ A2 Proposition (9.5)). In this case, ω1 and ω2 in (14) are one- to-one. Because ω1 + ω2 is bijective, every element s in Figure 4. Examples of spaces X ,X ⊆ R2 and their subspaces Hq(X1 ∪X2) can be uniquely represented by s1 +s2, where 1 2 s = ω (s ) for some s ∈ H (X ), i = 1, 2. In particular, A1,A2. Then β0(X1 \ A1) − β0(X1)+ β0(A1)=3 − 1+1=3 i i i i q i β X A β X β A we have thee followinge theorem. and 0( 2 \ 2) − 0( 2)+ 0( 2)=2 − 1+4=5. Theorem 2.3.1. Let X,X ,X , q ≥ 0 and ρ ,ω ,γ, i = 1 2 i i Definition. Let X,X ,X , and G , G , i = 1, 2 be de- 1, 2 be defined as above. If cl (X ) ∩ cl (X ) = ∅, then 1 2 1 2 X 1 X 2 fined as in Theorem 2.3.1, then the qth local merging num- the following hold: ber of X1 with respect to X2 in X is defined as the num- ber of barcodes in the multiset , denoted by (a) The persistence diagram Pq(G1) has no barcodes of (2, 3) Pq(G1) . birth = 2 if and only if Hq(X2)= {0}. mq(X1; X2) We describe the main differences between local merging (b) For s2 = ω2(s2) ∈ Hq(X1 ∪ X2), s2 6= 0 if and only if it is born at e2 in G . and local branch numbers ([64]). Take Figure 4 as an illus- 1 trative example. Based on [64], the local branch number of (c) For non-zero s2 ∈ Hq(X2), (s1, s2) ∈ Hq(X1) ⊕ A1 in Figure 4 is 4 since the graph which fits A1 has four e e e Hq(X2) is a local section in (12) for some s1 ∈ vertices in degree 1. On the other hand, the local merging e Hq(X1) if and only if s2 := ω2(s2) has barcode (2, 3) number of A1 in Figure 4 is 3 as demonstrated in Figure 7 e in G1. (g)-(i). As shown in Figure 4 (c), there are three connected components of X1 \ A1 that connect to A1. Another differ- Proof. (a) Because ω1 + ω2 is an isomorphism, Pq(G1) has ence is that the local branch numbers are constructed from not barcodes of birth = 2 if and only if im(ω1)= Hq(X1 ∪ Vietoris-Rips complexes on the point cloud data. There- X2) = im(ω1 + ω2) if and only if ω2 is the zero map if and fore, tuning parameters such as sampling the points, and only if Hq(X2)= {0} (since w2 is one-to-one). choosing the radius would be a critical step [64]. On the (b) The converse direction is trivial. If s2 = ω2(s2) = e other hand, our local merging numbers are purely depen- (ω1 + ω2)(0, s2) is non-zero, then s2 ∈/ im(ω1) = im(ω1 + e dent on the choices of X1,X2 ⊆ X and can be exactly ω2) since ω1 + ω2 is one-to-one. computed by the cellular sheaf structures. Branches and (c) By (b), s2 is born at 2 since s2 6= 0 and ω2 is one- e merging relations base on different geometric aspects, we to-one. To adapt Theorem 2.2.2, it is sufficient to note that thought both features are important for describing the con- ρ1(s1) = s2 if and only if γ(ω1(s1)) = s2 i.e., s1 and s2 e e e nection between global and local objects. coincide at 3 in G1. Now (c) follows from Theorem 2.2.2 immediately. Definition. A system of patches of a topological space X (i) (i) is a (finite) collection of pairs (X1 ,X2 )’s of subspaces As in Figure 5, the 0-dimensional PD P0(G1) of G1 (i) (i) of X such that clX (X1 ) ∩ clX (X2 )= ∅ for every i. is the multiset {(1, +∞), (2, 3), (2, 3)} while P0(G2) = {(1, +∞), (2, 3)}. It indicates that the space X1 has two Remark. As shown in Figure 4 (a)-(c), it may suggest that th merging parts in X. From the X2’s point of view, it has the 0 local merging number of A ⊆ X as single merging part since X1 is the unique component in X β0(X \ A) − β0(X)+ β0(A). (16) which connects to X2. Now the following definitions can be established: This is the special case when β0(A) = 1. In general, this 6 If X is a topological space and A is a subset of X, clX (A) denotes formula (16) would not hold. For example, in Figure 4 (d)- the closure of A in X. (f), β0(X2 \A2)−β0(X2)+β0(A2) equals 5 while the local (a) X (b) X1 (c) X2 (d) X1 ∪X2

(a) Input image (b) Heat map (c) Matrix 2 Figure 5. An example of spaces X in R and its subspaces X1,X2, which satisfy clX (X1) ∩ clX (X2)= ∅.

(d) Input image (e) Heat map (f) Matrix

(i) 9 (a) Input image (b) Windows (c) {X1 }i=1 (d) Merging numbers

Figure 6. An example of windows, patches and merging numbers.

(g) Input image (h) Heat map (i) Matrix merging number of A2 is 2. It shows that there are exactly two connected components in X2 that connect to A2 and gives us more explainable information than the number 5. Figure 7. More examples of images, heat maps, and matrices.

3. Demonstration

In the section, we demonstrate how do we use the pro- posed method to automatically detect local merging num- bers of a binary image. The demonstration aims to au- tomatically generated a heat map m0(f) for representing (a) Input image (b) Heat map local merging numbers of local regions of a binary im- age f : S → {0, 1} where S = ([a,b] × [c,d]) ∩ Z2. Figure 8. Image and its heat map of local merging numbers. Local Clearly, the m0(f) depends on how do we define patches parts with more complicated structures may have higher merging (i) (i) n −1 numbers (e.g. cross parts, corners or cusps). {(X1 ,X2 )}i=1 of f (0), hence m0(f) is not unique. To construct a system of patches of a binary image, for each local window S′ := ([a′,b′] × [c′,d′]) ∩ Z2 with a ≤ the matrix ′ ′ and ′ ′ , we define 1 1 1 a ≤ b ≤ b c ≤ c ≤ d ≤ d Xc1 = −1 ′ and −1 . Next, we define 1 8 1 (18) f (0) ∩ S X2 = f (0) \ Xc1 1 1 1 ′ ′ (17) X1 = Xc1 \{(x,y) ∈ Xc1 : x = a or y = b }. of local merging numbers. The black pixels separated by the central window have a merging number 8 since there In other words, we obtain X1 by removing all black pix- are 8 fans connect to the central object. Other fans have the els which belong to the boundary of S′. Because finite cu- same merging number 1 since only one connected object bic complexes in R2 are finite unions of closed R2-squares, connects to each fan. With the same windows in Figure 6 they are closed in R2 (and so do X). Hence we must have (b), we provide more heat maps and corresponding matrices clX (X1) ∩ clX (X2)= ∅. of squared images as examples for explaining our method. For example, as in Figure 6, if we consider local win- Every image was resized into 100 × 100 pixels, and its heat (i) 9 map can be generated in nearly 5 seconds7. dows in (b), then for image (a), the family {X1 }i=1 of black pixels in (c) induces a system of patches Alternatively, we can also apply the sliding window al- (i) (i) 9 {(X1 ,X2 )}i=1, and (d) is the corresponding heat map 7Local merging numbers are invariant under the transformation of sim- of local merging numbers. The heat map (d) corresponds to ilar objects in R2. cated sheaf structures, such as the coincidence of n-tuples (s1,...,sn) and their topological insights behind the alge- braic coincidence. Although the form of the diagram (6) is the simplest form of the zigzag homology [20, 57], multi-parameter persis- tent homology [20, 57], and cellular sheaf [24, 55, 54], the fruitful tools developed in the theories were not considered in the work, such as cellular sheaf cohomology and multi- persistence. Analyzing digital images by combining these advanced tools is also a future direction. We also demonstrate our method on UIUC dataset [45] in Figure 9, which is a well-known dataset in texture classi- fication task [41]. The UIUC dataset provides 25 classes of (a) Selected Images in UIUC images in different textures. We select one image per class and compute its heat map, where we transform each image to a binary one by setting its average pixel value as a thresh- old. The result shows that images in different classes have different characteristics of heat maps. For example, heat maps of textures with regular patterns may distribute more uniformly, and heat maps of natural textures (e.g. wood sur- faces) may contain small and non-regular pieces that have high m0(f). As a low-level feature, the local merging num- bers may provide useful information in certain image clas- sification tasks. Lastly, since the local merging numbers provide a heat map of an image, it can be viewed as a natural attention map. Local parts that have higher local merging values show that they are the joint areas crossed by different ob- (b) Heat maps of merging numbers of selected Images jects, and might be important in some image analysis tasks (e.g. medical images). Because every image and its heat Figure 9. Images in UIUC dataset and their heat maps. map have the same size, the latter one can be viewed as an additional channel of an image input in neural network models (e.g. AlexNet [43] or ResNet [37]). We also plan to gorithm [12] to compute the local merging numbers of local integrate this geometrically explainable feature into current windows. In the demonstration, we unify each binary image DNN models for some computer vision tasks. f to have size of 100 × 100 pixels, and consider three local To conclude this paper, we highlight our contribution. windows of sizes 10 × 10, 20 × 20, and 30 × 30. For each We proposed a new theory to generate the local merging n local window we obtain heat map m0 (f), n = 10, 20, 30, numbers in a binary image by using cellular sheaves on n and finally output m0(f) as the sum of all m0 (f)’s. Fig- short persistent homology. Moreover, the approximation ure 8 and Figure 9 are examples. The codes of the demon- theorem shows that local sections in cellular sheaves can stration were run by Matlab 2020b based on Windows be computed via barcodes in persistence diagrams of short 10, and the PDs were computed by the software Perseus filtrations. Because the local merging numbers are purely [52]. provided by the intrinsic geometry of the given image, no learning procedures needed in the method. We also im- 4. Discussion and Conclusion plemented the code for generating local merging numbers, which has the potential to be integrated into DNN models. Based on our investigation, the proposed work is the first one that combines sheaf structures and persistence diagrams Acknowledgements to capture the local merging numbers of binary images. We use the simplest sheaf structure (6) to derive the merging Chuan-Shen Hu is supported by the projects MOST 107- relation between pairs of local objects. One future direc- 2115-M-003-012-MY2, MOST 108-2119-M-002-031, and tion is to investigate the meaning and applications of local MOST 108-2115-M-003-005-MY2 hosted by the Ministry sections of homologies in q ≥ 1, and consider more compli- of Science and Technology in Taiwan. References [19] Gunnar Carlsson, Vin De Silva, and Dmitriy Morozov. Zigzag persistent homology and real-valued functions. In In [1] Henry Adams, Tegan Emerson, Michael Kirby, Rachel Proceedings of the 25th ACM Symposium on Computational Neville, Chris Peterson, Patrick Shipman, Sofya Chepush- Geometry, 2009. 2 tanova, Eric Hanson, Francis Motta, and Lori Ziegelmeier. [20] Gunnar Carlsson, Gurjeet Singh, and Afra Zomorodian. Persistence images: A stable vector representation of per- Computing multidimensional persistence. In Algorithms sistent homology. Journal of Machine Learning Research, and Computation, pages 730–739, Berlin, Heidelberg, 2009. 18(8):1–35, 2017. 1 Springer Berlin Heidelberg. 2, 8 ¨ [2] Paul Alexandroff. Uber den allgemeinen dimensionsbegriff [21] Gunnar Carlsson, Afra Zomorodian, Anne Collins, and und seine beziehungen zur elementaren geometrischen an- Leonidas Guibas. Persistence barcodes for shapes. Inter- schauung. Mathematische Annalen, 1928. 2 national Journal of Shape Modeling, 11:149–188, 01 2005. [3] Pavel Alexandroff. Diskrete raume¨ . Mat. Sb, 2:501-518, 1 1937. 2, 3 [22] Moo Chung, Jamie Hanson, Jieping Ye, Richard Davidson, [4] Pavel Alexandrov. Combinatorial Topology. Dover Publica- and Seth Pollak. Persistent homology in sparse regression tions, Inc., 31 East 2nd Street, Mineola, NY, 1947. 2, 3 and its application to brain morphometry. IEEE transactions [5] D Vijay Anand, Zhenyu Meng, Kelin Xia, and Yuguang Mu. on medical imaging, 34, 08 2014. 1 Weighted persistent homology for osmolyte molecular ag- [23] Justin Curry. Sheaves, Cosheaves and Applications. PhD gregation and hydrogen-bonding network analysis. Scientific Thesis, 2014. 3 Reports, 10:9685, 06 2020. 1 [24] Justin Curry. Topological data analysis and cosheaves. Japan [6] Ulrich Bauer, Michael Kerber, Jan Reininghaus, and Hubert Journal of Industrial and Applied Mathematics, 2015. 2, 3, Wagner. Phat - persistent homology algorithms toolbox. J. 8 Symb. Comput., 78:76–90, 2017. 1 [25] Herbert Edelsbrunner. Persistent homology: theory and prac- [7] Paul Bendich, James Marron, Ezra Miller, Alex Pieloch, and tice. Bulletin of the American Mathematical Society, 2014. Sean Skwerer. Persistent homology analysis of brain artery 1 trees. Arxiv, 2014. 1 [26] Herbert Edelsbrunner and John Harer. Computational Topol- [8] Nicolas Berkouk. Persistence and Sheaves : from Theory to ogy: An Introduction. American Mathematical Society, 01 Applications. Theses, Institut Polytechnique de Paris, Sept. 2010. 1, 4, 5 2020. 2 [27] Herbert Edelsbrunner, David Letscher, and Afra Zomoro- [9] Alexander Bernstein, Evgeny Burnaev, Maxim Sharaev, dian. Topological persistence and simplification. Discrete Ekaterina Kondrateva, and Oleg Kachan. Topological data Comput Geom, 28:511–533, 2002. 1 analysis in computer vision. In Twelfth International Con- [28] Samuel Eilenberg and Norman Steenrod. Foundations of ference on Machine Vision (ICMV), page 140, 01 2020. 1 Algebraic Topology. Foundations of Algebraic Topology. [10] Christophe Biscio and Jesper Møller. The accumulated per- Princeton University Press, 1952. 2 sistence function, a new useful functional summary statistic [29] Brittany Fasy and Bei Wang. Exploring persistent local ho- for topological data analysis, with a view to brain artery trees mology in topological data analysis. In 2016 IEEE Interna- and spatial point process applications. Journal of Computa- tional Conference on Acoustics, Speech and Signal Process- tional and Graphical Statistics, 28:671 – 681, 2016. 1 ing (ICASSP), pages 6430–6434, 03 2016. 2 [11] Holger Boche, Mijail Guillemard, Gitta Kutyniok, and [30] Brittany Fasy and Carola Wenk. Local persistent homology Friedrich Philipp. Signal analysis with frame theory and per- based distance between maps. In the 22nd ACM SIGSPA- sistent homology. The Conference of Sampling Theory and TIAL International Conference, pages 43–52, 11 2014. 2 Applications, SampTA’13, 01 2013. 1 [31] Robert Ghrist. Elementary Applied Topology. CreateSpace [12] Vladimir Braverman. Sliding Window Algorithms, pages Independent Publishing Platform, 2014. 2 2006–2011. Springer New York, New York, NY, 2016. 8 [32] Robert Ghrist and Hans Riess. Cellular sheaves of lattices [13] Glen Bredon. Sheaf Theory. Graduate Texts in Mathematics. and the tarski laplacian. Arxiv, 2020. 3 Springer New York, 1997. 2, 3 [33] Marvin Greenberg and John Harper. Algebraic Topology, A [14] Peter Bubenik. Statistical topological data analysis using First Course. Addison-Wesley Publishing Company, 1980. persistence landscapes. Journal of Machine Learning Re- 4, 5, 6 search, 16(3):77–102, 2015. 1 [34] Jakob Hansen and Robert Ghrist. Opinion dynamics on dis- [15] Gunnar Carlsson. Topology and data. Bulletin of the Ameri- course sheaves. Arxiv, 2020. 2 can Mathematical Society, 46(2):255–308, 2009. 1 [35] Heather Harrington, Nina Otter, Hal Schenck, and Ulrike [16] Gunnar Carlsson and Vin de Silva. Zigzag persistence. Arxiv, Tillmann. Stratifying multiparameter persistent homology. 2008. 2 Arxiv, 2019. 2 [17] Gunnar Carlsson, Vin de Silva, Sara Kalisnik,ˇ and Dmitriy [36] Robin Hartshorne. Algebraic Geometry. Springer-Verlag, Morozov. Parametrized homology via zigzag persistence. New York, 1977. 2, 3 Algebr. Geom. Topol., 19(2):657–700, 2019. 2 [37] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. [18] Gunnar Carlsson, Anjan Dwaraknath, and Bradley Nelson. Deep residual learning for image recognition. In 2016 IEEE Persistent and zigzag homology: A matrix factorization Conference on Computer Vision and Pattern Recognition viewpoint. Arxiv, 2019. 2 (CVPR), pages 770–778, 2016. 8 [38] Tomasz Kaczynski, Konstantin Mischaikow, and Marian [54] Michael Robinson. The nyquist theorem for cellular sheaves. Mrozek. Computational Homology. Applied Mathematical Sampling Theory and Applications 2013, Bremen, Germany, Sciences. Springer New York, 2004. 5 2013. 2, 3, 8 [39] Masaki Kashiwara and Pierre Schapira. Persistent homology [55] Michael Robinson. Topological Signal Processing. Mathe- and microlocal sheaf theory. Journal of Applied and Compu- matical Engineering. Springer Berlin Heidelberg, 2014. 2, 3, tational Topology, 2, 10 2018. 2 8 [40] Masaki Kashiwara and Pierre Schapira. Piecewise Linear [56] Michael Robinson. Hunting for foxes with sheaves. Notices Sheaves. International Mathematics Research Notices, 08 of the American Mathematical Society, page 661, 05 2019. 2019. rnz145. 2 2, 3 [41] Soheil Kolouri, Yang Zou, and Gustavo Rohde. Sliced [57] Sara Scaramuccia, Federico Iuricich, Leila De Floriani, and wasserstein kernels for probability distributions. In IEEE Claudia Landi. Computing multiparameter persistent homol- Conference on Computer Vision and Pattern Recognition ogy through a discrete morse-based approach. Computa- (CVPR), pages 5258–5267, 06 2016. 8 tional Geometry, 89:101623, 2020. 2, 8 [42] Rami Kraft. Illustrations of Data Analysis Using the Mapper [58] Jean Serra. Image Analysis and Mathematical Morphology. Algorithm and Persistent Homology. TRITA-MAT-E. Master Number 1 in Image Analysis and Mathematical Morphology. Thesis, 2016. 2 Academic Press, 1984. 5 [43] Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton. Ima- [59] Gurjeet Singh, Facundo Memoli,´ and Gunnar Carlsson. genet classification with deep convolutional neural networks. Topological methods for the analysis of high dimensional In F. Pereira, C. J. C. Burges, L. Bottou, and K. Q. Wein- data sets and 3d object recognition. In PBG@Eurographics, berger, editors, Advances in Neural Information Processing 2007. 2 Systems, volume 25, pages 1097–1105. Curran Associates, [60] Pierre Soille. Morphological Image Analysis: Principles and Inc., 2012. 8 Applications. Springer-Verlag New York, Inc., Secaucus, NJ, [44] Genki Kusano, Kenji Fukumizu, and Yasuaki Hiraoka. Ker- USA, 2 edition, 2003. 5 nel method for persistence diagrams via kernel embedding [61] Anirudh Som, Hongjun Choi, Karthikeyan Natesan Rama- and weight factor. Journal of Machine Learning Research, murthy, Matthew Buman, and Pavan Turaga. Pi-net: A deep 18(189):1–41, 2018. 1 learning approach to extract topological persistence images. [45] Svetlana Lazebnik, Cordelia Schmid, and Jean Ponce. A CoRR, abs/1906.01769, 2019. 1 sparse texture representation using local affine regions. IEEE [62] Daniel Spitz, Jurgen¨ Berges, Markus Oberthaler, and Anna Transactions on Pattern Analysis and Machine Intelligence, Wienhard. Finding universal structures in quantum many- 27(8):1265–1278, 2005. 8 body dynamics via persistent homology. Arxiv, 2020. 1 [46] Hyekyoung Lee, Hyejin Kang, Moo Chung, Bung-Nyun [63] The GUDHI Project. GUDHI User and Reference Manual. Kim, and Dong Soo Lee. Persistent brain network homol- GUDHI Editorial Board, 3.1.1 edition, 2020. 5 ogy from the perspective of dendrogram. IEEE Transactions [64] Robin Vandaele, Tijl De Bie, and Yvan Saeys. Local topo- on Medical Imaging, 31(12):2267–2277, 2012. 1 logical data analysis to uncover the global structure of data [47] Qing Liu. Algebraic Geometry and Arithmetic Curves. Ox- approaching graph-structured . In Machine Learn- ford Graduate Texts in Mathematics, 2006. 2, 3 ing and Knowledge Discovery in Databases, pages 19–36, [48] Clement´ Maria. Filtered complexes. In GUDHI User and Cham, 2019. Springer International Publishing. 2, 6 Reference Manual. GUDHI Editorial Board, 3.1.1 edition, [65] Hubert Wagner, Chao Chen, and Erald Vuc¸ini. Efficient 2020. 5 Computation of Persistent Homology for Cubical Data. [49] Clement´ Maria, Jean-Daniel Boissonnat, Marc Glisse, and Springer Berlin Heidelberg, Berlin, Heidelberg, 2012. 5 Mariette Yvinec. The gudhi library: Simplicial complexes [66] Zhichao Wang, Qian Li, Gang Li, and Guandong Xu. Poly- and persistent homology. In Mathematical Software – ICMS nomial representation for persistence diagram. In 2019 2014, pages 167–174, Berlin, Heidelberg, 2014. Springer IEEE/CVF Conference on Computer Vision and Pattern Berlin Heidelberg. 1 Recognition (CVPR), pages 6116–6125, 2019. 1 [50] Gabriell Mat´ e,´ Andreas Hofmann, Nicolas Wenzel, and Di- [67] Xiaojin Zhu. Persistent homology: An introduction and a eter Heermann. A topological similarity measure for pro- new text representation for natural language processing. IJ- teins. Biochimica et Biophysica Acta (BBA) - Biomembranes, CAI International Joint Conference on Artificial Intelligence, 1838(4):1180 – 1190, 2014. Viral Membrane Proteins - pages 1953–1959, 08 2013. 1 Channels for Cellular Networking. 1 [51] Laurent Najman and Hugues Talbot. Mathematical Morphol- ogy. Wiley-ISTE, 1d edition, 2010. 5 [52] Vidit Nanda. Perseus, the persistent homology soft- ware. http://www.sas.upenn.edu/˜vnanda/ perseus, 2013. 8 [53] Chi Seng Pun, Kelin Xia, and Si Xian Lee. Persistent- homology-based machine learning and its applications – a survey. Arxiv, 2018. 1