A New Order Theory of Set Systems and Better Quasi-Orderings

Progress in Informatics, No. 9, pp.9–18, (2012) 9 Special issue: Theoretical computer science and discrete mathematics

Research Paper A new order theory of set systems and better quasi-orderings

Yohji AKAMA1 1Mathematical Institute, Tohoku University

ABSTRACT By reformulating a learning process of a set system L as a game between Teacher (presenter of data) and Learner (updater of abstract independent set), we define the order type dim L of L to be the order type of the game tree. The theory of this new order type and continuous, monotone function between set systems corresponds to the theory of well quasi-orderings (WQOs). As Nash-Williams developed the theory of WQOs to the theory of better quasi- orderings (BQOs), we introduce a set system that has order type and corresponds to a BQO. We prove that the class of set systems corresponding to BQOs is closed by any monotone function. In (Shinohara and Arimura. “Inductive inference of unbounded unions of pattern languages from positive data.” Theoretical Computer Science, pp. 191–209, 2000), for any set system L, they considered the class of arbitrary (finite) unions of members of L.Fromview- point of WQOs and BQOs, we characterize the set systems L such that the class of arbitrary (finite) unions of members of L has order type. The characterization shows that the order structure of the set system L with respect to the set inclusion is not important for the result- ing set system having order type. We point out continuous, monotone function of set systems is similar to positive reduction to Jockusch-Owings’ weakly semirecursive sets.

KEYWORDS Better elasticity, continuous deformation, powerset orderings, linearization, unbounded unions, wqo

1 Introduction ory. In [22], [29], [39], sufficient conditions for set sys- A set system L over a set T, a subfamily of the pow- tems being learnable is studied with König’s lemma erset P(T), is a topic of (extremal) combinatorics[5], and Ramsey’s theorem, In [38], for a set system L, [21], as well as a target of an algorithm to learn in com- Shinohara-Arimura considered the unbounded unions <ω putational learning theory [26]. of L, that is, the class L of nonempty finite unions of A well quasi-ordering [16] (WQO for short) is, by members of L, and then they used Higman’s theorem to definition, a quasi-ordering (X, ) which has nei- study a sufficient condition for it being learnable. In [9], ther an infinite antichain nor an infinite descending de Brecht employed WQOs to calibrate mind change chain. WQOs are employed in algebra [16], combina- complexity of unbounded unions of restricted pattern torics [23], [35], formal language theory [6], [7], [14], languages. Motivated by [22], [29], [39], a somehow [32], and so on. systematic study on the relation between WQOsanda WQOs and related theorems such as Higman’s the- class of learnable set systems is done in [3], as follows: orem [16], König’s lemma and Ramsey’s theorem [5] are sometimes employed in computational learning the- (i) By reformulating a learning process of a set system L as a game between Teacher (presenter of data) Received October 31, 2011; Revised January 10, 2012; Accepted January 10, 2012. and Learner (updater of abstract independent set), we 1) [email protected] define the order type dim L of L to be the order type DOI: 10.2201/NiiPi.2012.9.3

c 2012 National Institute of Informatics 03

10 Progress in Informatics, No. 9, pp.9–18, (2012)

of the game tree, if the tree is well-founded. According The class of WQOs is closed under finitary operations to computational learning theory, if an indexed family such as Higman embedding [16] and topological minor L of recursive languages has well-defined dim L then relation [13 Sect. 1.7] between finite trees [13 Ch. 12], L is learnable by an algorithm from positive data. If [23]. The class of finite graphs is a WQO under the mi- a set system has the well-defined order type, then we nor relation. Robertson-Seymour’s proof of it is given callitafinitely elastic set system (FESS for short). See in the numbers IV-VII, IX-XII and XIV-XX of their se- Definition 12. ries of over 20 papers under the common title of Graph Minors, which has been appearing in the Journal of (ii) For each quasi-ordering X = (X, ), we consider Combinatorial Theory, Series B, since 1983. For a the set system ss (X) consisting of upper-closed subsets shorter proof, see recent papers by Kawarabayashi and of X. The set system has the order type equal to the his coauthors. maximal order type [12] of X. Furthermore, the con- The class of FESSs enjoys a useful closure condition: struction ss (•) has a left-inverse qo (·). Hereforaset L M L (L) L, L Proposition 1 ( [3]) For any set systems and and system ,qo is a quasi-ordering ( )such M that any continuous deformation O : {0, 1} →L,ifM is an FESS, so is the image O [M] of M by O. x L y ⇐⇒ ∀ L ∈L, (x ∈ L =⇒ y ∈ L) . By it, we prove that for various (nondeterminis- The maximal order type otp(X)ofX is defined if and tic) language operators (e.g. Kleene-closure, shuffle- only if X is a WQO. For any quasi-ordering X, if one of product [34], [37], shuffle-closure [17], (iterated) literal otp(X) and dim ss (X) is defined then the other side is shuffle [4], union, product, intersection), the element- defined with the same ordinal number. So FESSs corre- wise application of such operator to (an) FESS(s) in- spond to WQOs. duces an FESS. Roughly speaking, a deformation transforms any ∀ (iii) For every nonempty set U, the product topologi- quasi-ordering to the powerset ordering [28] ∃. cal space {0, 1}U of the discrete topology {0, 1} is called It is through our correspondence (ss (•) , qo (·))be- a Cantor space. A subspace of a Cantor space is repre- tween quasi-orderings to set systems (Section 3). Al- sented by L, M,....WesayafunctionfromM to L is though the powerset ordering of Rado’s WQO [33] is continuous, if it is continuous with respect to the sub- not a WQO [28 Corollary 12], the class of better quasi- spaces M, L of the Cantor spaces. We identify {0, 1}U orderings [31] (BQOs for short) is closed with respect with the powerset P(U), and a function from {0, 1}U to to the powerset ordering. There are infinitary opera- {0, 1}U with a function from P(U)toP(U). We say tions under which the class of WQOs is not closed but O : M→Lis a deformation, if it is monotone (i.e. the class of BQOs is. So we introduce a better elastic M ⊆ M implies O(M) ⊆ O(M ).) If a deformation is set system (BESS for short), as a set system correspond- continuous, then it has following finiteness condition: ing to a BQO. We show that the class of BESSsisclosed under the image of any deformation (Section 3), where Lemma 1 Let O : {0, 1} M →L. any deformations are infinitary in a sense of Lemma 1. 1. O is a deformation, if and only if there is a binary By this and (i), we can develop the computational learn- relation R ⊆ ( L) × P ( M) such that ing theory of ω-languages [36]. The notion of BESS is useful in investigating a fol- ∀M ∈M∀x ∈ L lowing set system O ⇔∃ ⊆ . , . (M) x v M R(x v) (1) L<ω := { M ; ∅ M⊆L, #M < ∞}. (2) 2. O is a continuous deformation, if and only if there It is studied for the learnability of language classes such exists R ⊆ ( L) × P ( M) such that (1) holds, but as a class of regular pattern languages [38]. We charac- visafinite set whenever R(x, v) holds, and there are <ω terize the set systems L such that L are again FESSs, only finitely many such v’s for each x. ( [3]) <ω and then we prove that for every BESS L, L is an For each binary relation R ⊆ ( L) × P( M),the FESS. function O satisfying (1) is unique. So we write it by M We remark that another importance of set system OR. Conversely, every deformation O : {0, 1} →L <ω L . We conjecture that the order type of an FESS is the is written as O by a binary relation R supremum, actually the maximum, of the order types of the “linearizations” of the FESS. This conjecture corre- R := (x, v) ∈ L × P M ; O(v)(x) = 1 . sponds to a proposition useful in investigating WQOs: 03

A new order theory of set systems and better quasi-orderings 11

Proposition 2 ( [12]) The order type of a WQO is the of U such that the cardinality of X is less than α (equal <α α maximum order type of the linearizations of the WQO. to α, resp.) is denoted by [U] ([U] , resp.). A set X ⊆ ω is often identified with the sequence enumerating it L A “linearization” of an FESS seems to be a subfamily in a strictly increasing order. of L<ω. <ω We hope that our study on FESSsandBESSsareuse- Definition 1 1. We say B ⊆ [ω] abarrier,if(1)aset ful in solving problems (e.g. decision problem of timed B is infinite; (2) for all σ ∈ [ B]ω there exists Petri-nets [1], [2], the multiplicative exponential linear s ∈ B such that s is a prefix of σ; and (3) for all logic [11]) which are related to WQOsandBQOsbut s, t ∈ B, s t. <ω hard to solve with conventional arguments for WQOs 2. For s, t ∈ [ω] , we write s t, if, the sequence s is and BQOs. a prefix of the sequence u = s ∪ t and the sequence The rest of paper is organized as follows: In the next tisaprefixofu\{min u}. section, we recall the powerset ordering and Marcone’s 3. Let o.t.(B) the maximal order type of B with respect characterization [28] of WQOs such that the powerset to the lexicographical ordering. orderings are again WQOs. We also review the combinatorial definition of a BQO. Then we recall that the Observe that class of BQOs is closed with respect to the powerset or- Singl := {{n} ; n ∈ ω} (3) dering. In Section 3, for the set system ss () of upper- closed sets of a fixed quasi-ordering ,theimageby is a barrier. Any barrier B of o.t.(B)beingω consists a deformation is essentially the set system of upper- only of singletons, according to [28 p. 342]. closed sets of the powerset ordering of . Then we in- We recall an α-WQO and a BQO [28 Definition 3]. troduce a BESS as an FESS that corresponds to a BQO. Then we prove that the image of a BESS by any defor- Definition 2 Let α be a countable ordinal and a mation is an FESS. In Section 4, we characterize the quasi-ordering on Q. We say a function f : B → Q <ω class of set systems L such that L is an FESS, from is good with respect to , if there are some s, t ∈ Bsuch viewpoint of BQO theory. We contrast our characteriza- that st and f (s) f (t). Otherwise we say f is bad.We tion with Shinohara-Arimura’s sufficient condition [38] say an α-WQO, if for every barrier with o.t.(B) ≤ α <ω for a set system L to have an FESS L . every function f : B → Q is good with respect to .If In appendix, we propose to extend Ramsey numbers is an α-WQO for each countable ordinal α,wecall to estimate the ordinal order type of set systems, and a BQO. then present miscellaneous results for computable ana- logue of (iii). Finally, we review the order type of a set Because Singl is a barrier, every BQO is a WQO. system from [3]. When we define an α-WQO for a countable ordinal α, we have only to consider only smooth barriers among 2 Better quasi-orderings and powerset the barriers, as [28] explains: ordering A better quasi-ordering (BQO for short), a stronger Definition 3 By a smooth barrier, we mean a barrier B , ∈ < ≤ concept than a WQO, has pleasing closure properties such that for all s t B with #s #t there exists i #s with respect to such that the i-th smallest element of s is less than that of t. • embedding for transfinite sequences [31]; By an indecomposable ordinal, we mean ωβ such • topological minor relation for infinite trees that β is any ordinal. Recall [27 Corollary 3.5]. [24], [30]; and Proposition 3 If B is a smooth barrier, then the ordinal ∀ . . • a powerset ordering ∃ (see Definition 4) ( [28 o t (B) is indecomposable. Corollary 10]). Then we have [28 Theorem 4]. 2.1 Combinatorial definition of BQOs Proposition 4 Let α be a countable ordinal and a We first recall the definition of BQOsbybarriers [28] quasi-ordering on Q. is an α-WQO if and only if for and then the closure of BQOs with respect to the pow- every smooth barrier B with o.t.(B) ≤ α, every map ∀ erset ordering ∃. Please be advised to refer [27], [28] f : B → Qisgood with respect to . for the detail. Hereafter the first infinite ordinal ω is identified with Corollary 1 is a BQO if and only if it is an α-WQO the set of nonnegative integers. The class of subsets X for each countable infinite indecomposable ordinal α. 03

12 Progress in Informatics, No. 9, pp.9–18, (2012)

We use properties of a barrier from [28 Lemma 6]. 3 Better elasticity — deformation as powerset ordering <ω Proposition 5 For a barrier B, let B2 ⊆ [ω] be Given a deformation O and a WQO X, we try to con- struct explicitly from X a suitable WQO Y such that B2 := {s ∪ t ; s, t ∈ B, s t}. O[ss (X)] ⊆ ss (Y).

Then Definition 5 Let L and M be set systems. Suppose 2 1. for each t ∈ B there exist unique π0(t),π1(t) ∈ B R ⊆ ( L) × P ( M) and X := ( M, ) is a π π = π ∪ π such that 0(t) 1(t) and t 0(t) 1(t); quasi-ordering. Then define a quasi-ordering QR(X) 2 2. if t, t ∈ B and t t then π1(t) = π0(t ); by ( L, ) as follows: For any x, x ∈ L, we write 3. B2 is a barrier; and x x , if whenever R(x, v) holds, there exists v such . . . . 2 = . . · ∀ 4. if o t (B) is indecomposable then o t (B ) o t (B) that R(x , v ) and v ∃ v . ω. Lemma 3 Let L and M be set systems. Suppose R ⊆ 2.2 Powerset ordering ( L) × P ( M).IfX is a quasi-ordering on M, Definition 4 For a quasi-ordering (X, ), we define a then QR(X) is indeed a quasi-ordering on L. quasi-ordering on the powerset P(X) by Proof. Let QR(X)be( L, ). It is easy to see that ∀ v ∃ v : ⇐⇒ ∀ x ∈ v ∃x ∈ v. x x . x x.Fromx x and x x ,wederivex x .Let R(x, v). By x x , there exists v such that R(x , v )and ∀ , It is studied for the reachability analysis of Petri v ∃ v .Byx x , there exists v such that R(x v ) ∀ ∀ nets (verification of infinite-state systems [1], Timed and v ∃ v . Because ∃ is a quasi-ordering, we have ∀ Petri net [1], [2],....) v ∃ v . Q.E.D.

Proposition 6 ( [18], [28 Theorem 9]) If α is a count- It seems difficult to replace a “quasi-ordering” with able infinite indecomposable ordinal and (Q, ) is an a “partial ordering” in Lemma 3. For a following the- ∀ · (α · ω)-WQO, then the powerset P(Q) ordered by ∃ is orem, see (ii) in the first section for a left-inverse qo ( ) an α-WQO. of ss (•), and Lemma 1 for the definition of OR. ⊆ L × M Lemma 2 For a quasi-ordering X = (Q, ), the fol- Theorem 1 For any R ( ) P ( ) , we have O M ⊆ M lowing are equivalent: R [ ] ss QR(qo ( )) . 1. (a) X is a WQO; and (b) let F be any function from Proof. Let ( M, ) = qo (M) and ( L, ) = [ω]2 to Q. Then if F({i, j}) ≺ F({i, j + 1}) for any QR(qo (M)). i < j <ω, then there are i < j < k <ωsuch that We verify if A ∈Mand OR(A) x x ,then F({i, j}) ≺ F({ j, k}). OR(A) x . 2. X is an ω2-WQO . <ω ∀ By OR(A) x, there exists v such that R(x, v)and 3. [Q] , is a WQO. ∃ v ⊆ A.Sincex x , there exists v such that R(x , v ) ∀ 4. P (Q) , ∃ is a WQO. and every x ∈ v has x ∈ v with x x . Because x ∈ A and A ∈Mis upper-closed with respect to , x ∈ A.

Proof. The proof is similar to that of [28 Corollary 12], So v ⊆ A. This means OR(A) x . Q.E.D. but we use Rado’s characterization [33 Theorem 3] of WQOs(Q, ) such that the set of sequences of elements We are not sure whether QR(qo (ss (X)))isaWQO of Q with the length ω is again a WQO.ForanyWQO for each WQO X, because, according to Lemma 2, the α <α ∀ X = (Q, ) and any countable ordinal α,letX (X quasi-ordering ∃ is not always a WQO. resp.) be the set of sequences of elements of Q of Instead, we introduce a stronger set system than an length α (less than α, resp.) quasi-ordered by naturally FESS. generalized Higman’s embedding. The condition (1) is ω Deﬁnition 6 (BESSs) We say a set system L⊆P(T) equivalent to X being a WQO, by [33 Theorem 3]. By ω <ω <ω2 is a better elastic set system (BESS for short) or L has Higman’s theorem, it is equivalent to (X ) = X better elasticity,providedqo (L) is a BQO. being a WQO. By [27], it is equivalent to (2). The equivalence to the other conditions follows from [28 Corol- Example 1 For every WQO X which is not a BQO,aset lary 12]. Q.E.D. system ss (X) is an FESS but not a BESS,sinceqo (·) is a left-inverse of ss (•). 03

A new order theory of set systems and better quasi-orderings 13

Lemma 4 1. A quasi-ordering (X, ) is a BQO, if and By Proposition 3 and Proposition 5 (4), o.t.(B2) = only if ss ((X, )) is a BESS. o.t.(B) · ω ≤ α · ω. Because B2 is a barrier by Proposi- ∀ 2. Every BESS is an FESS. tion 5 (3), (P( L), ∃) is not an (α · ω)-WQO,which contradicts Proposition 6. Q.E.D. Proof. (1) By (ii) in the first section. (2) If a set system L has an infinite bad learning sequence (see Defi- A following immediate corollary of Theorem 2 may nition 12) t0, L1, t1, L2,...,thenwehaveabarrier be useful in developing computational learning theory ∞ Singl and a function f ; {i}∈Singl → ti such that of ω-languages [36]: Let Σ be a set of possibly in- ti ∈ L j t j hence ti L t j (i < j). This contradicts that finite sequences of elements in an alphabet Σ,andL ∞ qo (L) is a BQO. Q.E.D. be a set system over Σ . The concatenation operation of two sequences is defined similarly as that of two fi- As the class of BQOs is closed under infinitary oper- nite sequences except that for an infinite sequence u and ations than the class of WQOs is, we prove that the class sequence v, the concatenation uv is defined as u.For ∞ ω of BESSs is closed under infinitary operations (i.e., de- L ⊆ Σ ,theω-closure [36] L of L is the set of in- formations) than the class of FESSs is. See Lemma 1 finitely iterated concatenation u1u2u3 ··· of sequences for the characterization of deformations. u1, u2,...∈ L. ω L Theorem 2 Assume L and M are set systems and O : Corollary 2 (Closure of -languages) If is a BESS, M→Lis a deformation. Then if M is a BESS,sois so are following classes: { ω ∈L} O [M]. 1. L ; L . 2. {Lsh ; L ∈L}.HereLsh is the shuffle-closure {ε}∪

To prove Theorem 2, we have only to verify a fol- L ∪ (L L) ∪ ((L L) L) ∪···, and L L is the set

lowing lemma, in view of Corollary 1: of u1v1u2v2 ··· such that u1u2 ··· ∈ L, v1v2 ··· ∈ L ∗ and ui, vi ∈ Σ (i ≥ 1). Lemma 5 Let O be a deformation from a set system M to a set system L.Ifα is a countable infinite inde- 4 Linearizations of set systems and composable ordinal, and qo (M) is an α·ω2-WQO,then powerset orderings qo (O[M]) is an α-WQO. Many studies on WQOs use de Jongh-Parikh’s theorem [12]: “The order type otp(X)ofaWQO X is the O O ⊆ L × Proof. Write as R for some relation R ( ) maximum of order types of the linearizations of X.” P ( M). Let the quasi-ordering qo (M) be ( M, ) We wish to require that if a linear order Y is a lin- and QR(qo (M))be( L, ), as in Definition 5. As- earization of a quasi-ordering of X,thenss(Y) is a sume qo (O[M]) is not an α-WQO. By Theorem 1 and ‘linearization’ of a ss (X). Then a ‘linearization’ of a set (ii) in the first section, system L⊆P(T) should be a set system M⊆P(T) linearly ordered by set inclusion and M consists of unions qo (O [M]) ⊇ qo ss Q (qo (M)) = Q (qo (M)). R R of members of L such that for any L ∈Lthere exists a subfamily L ⊆Lsuch that L ∈L and L ∈M. Since qo (O[M])is not an α-WQO, the quasi-ordering M = L, L L QR(qo ( )) ( ) is neither. By Proposition 4, Lemma 6 For any FESS ,ifqo ( ) is a WQO then the there are smooth barrier B of o.t.(B) ≤ α and a function class Lˆ := { M ; ∅ M⊆L}containing L closed f : B → O[M] such that for all u, v ∈ B, uv implies under arbitrary unions is an FESS. f (u) f (v). By Proposition 5 (1), for all t ∈ B2,we Proof. Since each member M∈Lˆ is upper- have π0(t),π1(t) ∈ B and π0(t) π1(t). So f (π0(t)) 2 closed with respect to the WQO qo (L), any learning f ((π1(t)))forallt ∈ B . By Definition 5, for some sequences (see Definition 12) of the set system Lˆ are v with R ( f (π0(t)) , v) and for all v ,ifR ( f (π1(t)) , v ) ∀ those of the set system ss qo (L) .Sinceqo(L) is a then v ∃ v . For each t,letg(t) be one of such v.Then WQO,ss qo (L) is an FESS. Therefore any bad learn- g is a function from B2 to P( M). ing sequence should be finite. Hence Lˆ is an FESS. Then for all t, t ∈ B2,wehaveg(t) ∀ g(t ) whenever ∃ Q.E.D. t t . In other words, the function g is bad with respect ∀ to the powerset ordering ∃. To verify it, first recall

π1(t) = π0(t ) by Proposition 5 (2). By the deﬁnition of Conjecture 1 Deﬁne a suitable linearization of a set ∀ g,wehaveg(t) ∃ v whenever R ( f (π1(t)) , v ).More- system. Do we have

over R ( f (π0(t )), g(t )). Because Proposition 5 (2) im- dim X = max {dim Y ; Y is a linearization of X} plies π1(t) = π0(t ), we have R ( f (π1(t)) , g(t )).There- ∀ fore, g(t) ∃ g(t ). for any set system X ? 03

14 Progress in Informatics, No. 9, pp.9–18, (2012)

<ω We characterize the set systems L such that the set Corollary 3 If L is a BESS, then both of L and Lˆ system of arbitrary (finite) unions of members of L is are FESSs. an FESS, from viewpoint of WQOs, BQOs, and the pow- ∀ L L erset ordering ∃. Proof. As is a BESS, the quasi-ordering qo ( ) is a BQO by the definition, and hence is an ω2-WQO,bythe Theorem 3 For any set system L, the following are definition of BQO. By Lemma 2, we have the condi- equivalent: tion (1) of Theorem 3 and thus the desired conclusions. 1. the quasi-ordering qo (L) = ( L, ≤) satisfies the Q.E.D. condition (1) of Lemma 2. Namely, (a) qo (L) is a WQO; and As the class of BQOs enjoys the closure properties (b) Let F be any function from [ω]2 to L.Thenif with respect to possibly infinitary constructions, we F({i, j}) < F({i, j + 1}) for any i < j <ω,then conjecture a following: there are i < j < k <ωsuch that F({i, j}) < <ω F({ j, k}). Conjecture 2 If L is a BESS, then both of L and Lˆ <ω 2. L is an FESS. are BESSs. 3. Lˆ is an FESS. We contrast our characterization of set systems L <ω Proof. Write qo (L) = ( L, ). Assume the having an FESS L , with Shinohara-Arimura’s suffi- condition (3) is false. Then there are an infinite se- cient condition [38] for a set system L to have an FESS M ⊆L = , ,... L<ω. quence n (n 1 2 ) and an infinite sequence x ∈ L (n = 1, 2,...) such that n L Definition 7 Let be a set system over X. L is said to have a finite thickness (FT),provided vn := {x1,...,xn−1}⊆ Mn xn. (4) that for any x ∈ X #{L ∈L; x ∈ L} < ∞. L is said ∀ to have no-infinite-antichain property (NIA),provided If there are n < m such that v v ,thenx x − n ∃ m i m 1 that L has no infinite antichain with respect to the set- for some i < n. However, (4) implies xi ∈ Mm−1 ∈ inclusion ⊆. xm−1. By the definition of , xi xm−1 implies xi Mm−1 xm−1. A contradiction. Thus the powerset ∀ The set system Singl has an FT but not NIA.Ifaset ordering (P( L), ) is not a WQO. By Lemma 2, the ∃ system has an FT,thenitisanFESS [38]. condition (1) is false. <ω Assume the condition (1) is false. Since Lemma 2 Proposition 7 ( [38]) If L has an FT and NIA,thenL implies that the condition (1) is equivalent to the well- is an FESS. ∀ quasi-orderedness of the powerset ordering ∃,wehave an infinite sequence (v ) such that for all i < j, v ∀ v However the conjunction of FT and NIA is not preserved i i i ∃ j <ω j · but vi ⊆ L. Then there is y ∈ v j such that for all by the operation ( ) . i ∈ i j y vi we have y y . By the definition of ,thereisa L = { , ∞ ∩ Z ≥ i ∈ j Lemma 7 1. A set system 2 [i ) ; i sequence (Li, j)i< jsuch that y Li, j y . Hence the se- <ω 1}∪{{0}} has an FT and NIA but L3 := (L2 ) is quence < Li, j is an infinite bad learning sequence i j j an FESS without an FT ([8]). <ω <ω in L .SoL is not an FESS. Thus the condition (2) 2. The converse of Proposition 7 is false. Actually, <ω is false. (L3 ) is an FESS but L3 does not have an FT. Obviously the condition (3) implies the condition (1). The equivalence between the condition (1) and the con- Lemma 8 1. For any FESS L, L has NIA if and only dition (2) can be similarly proved. Q.E.D. if (L, ⊇) is a WQO. <ω <ω 2. If L is an FESS,thenL has NIA.

Example 2 By [29], [39], we have an FESS Proof. (1) The if-part is immediate from the definition of WQOs. Assume there is an infinite descend- L1 := {{i}∪{k ; k ≥ j} ; i, j ∈ ω} , ing chain (Li)i ⊆Lwith respect to ⊇. Hence, L1 ··· ∈ \ because it is the memberwise union of an FESS {{k ∈ L2 L3 . By putting xi Li+1 Li,wehave <ω x , L , x , L , ··· is an infinite bad learning se- ω ; k ≥ j} ; j ∈ ω } and an FESS Singl . But (L1 ) 1 2 2 3 quence. This contradicts that L is an FESS. is not an FESS according to [9 Proposition 2.1.27]. The last assertion is an easy corollary of Theorem 3, be- M M ⊆L = , ,... (2) Suppose ( n)n ( n ;n 1 2 )is cause qo (L ) = (ω, =) and is not a WQO. L<ω M 1 an infinite antichain in .Then n

A new order theory of set systems and better quasi-orderings 15

<ω a strictly ascending chain in L , which is an FESS. (2)ByLemma8(1)andtheassertion(1)ofthis Q.E.D. Lemma, the partial ordering (L1, ⊇)isaWQO. (3) It is because a following function F does not sat- Following relation holds among (continuous) defor- isfy Lemma 2 (1) (b): mations, NIA and FT: F({i, j}):= {i}∪{k ; k j}. (i < j) Lemma 9 1. If a set system L has NIA, so does O[L] of L for any deformation O. (4) Let (ω, )beqo(L1).Thenn m (n < m) because ∈ 2. For any nonempty set X and for any x X, a func- n ∈ F({n, m + 1}) m, while n m (n > m) because tion O : P(X) → P(X); A → A ∪{x} is a contin- n ∈ F({n, n + 1}) m. Therefore qo (L1) is not a WQO, uous deformation. Thus even if L has an FT, O[L] hence is not a BQO.SoL1 does not satisfy the condi- does not. <ω tion 1 of Theorem 3. Hence (L1 ) is not an FESS. {O } { } ⊆ By Fact 1 (2) and the assertion (2) of this Lemma, Proof. (1) If the image (Li) i of Li i P(T) | L , ⊇ | L , ⊇ by a deformation O is an infinite antichain, then, for qo ( PF ( ( 1 ) ) ) is ( 1 ) which is not a BQO by the assertion (3) of this Lemma. any distinct i, j ∈ ω there exists ni, j ∈ O(Li) \ O(L j). O ∃ ∈ Let be as in the equation (1). Then we have vi, j (5) Since qo (L1) is not a WQO by the proof of the <ω P(Li). R(ni, j, vi, j)and∀v ∈ P(L j). ¬R(ni, j, v). There- assertion (4) of this lemma, (L1 ) is not an FESS be- fore vi, j is not a subset of L j. Thus, there exists ai, j ∈ cause of Theorem 3. By Fact 1 (1), the set system <ω vi, j \ L j ⊆ Li \ L j. Hence, {Li}i is also an infinite |PF ((L1, ⊇))| is ss ((L1, ⊇)), which is an FESS by the antichain. (2) It is immediate. Q.E.D. assertion (2) of this Lemma. Q.E.D. | L , ⊇ | Finally, we remark that a condition for a set system Although PF (( 1 )) is not a BESS, it satisfies the <ω L to satisfy L being an FESS does not depend on the condition (1) of Theorem 3, according to the assertion structure of L with respect to the set-inclusion, in view (5)ofLemma10. of the assertion (2) and the assertion (5) of following Lemma 10. We recall that a quasi-ordering X = (X, )isaWQO, Acknowledgement. if and only if any upper-closed subset of X is a finite This work is partially supported by Grant-in-Aid for union of principal filters [16]. Scientific Research (C) (21540105) of the Ministry of Education, Culture, Sports, Science and Tech- Definition 8 For a quasi-ordering X = (X, ), define nology (MEXT). The author thanks Hiroki Arimura, |PF (X)| to be the set of principal filters of X.LetPF (X) Masami Ito, Makoto Kanazawa, Mizuhito Ogawa, and be |PF (X)| ordered by the reverse set-inclusion. anonymous referees. Special thanks go to Ken-ichi Kawarabayashi for his encouragement after the 11- Fact 1 Let X be a quasi-ordering. March earthquake. <ω 1. ss (X) = |PF (X)| if and only if X is a WQO. 2. X = qo ( |PF (X)| ). References 3. X is order-isomorphic to PF (X) for any partial or- [1] P. A. Abdulla and A. Nylén. “Better is better than well: dering X. on efficient verification of infinite-state systems,” In 4. dim |PF (X)|≤dim ss (X) = otp(X). 15th Annual IEEE Symposium on Logic in Computer 5. For the partial order Science (Santa Barbara, CA, 2000), pp. 132–140. IEEE Comput. Soc. Press, Los Alamitos, CA, 2000.

X = ({b}∪{ai ; i ∈ ω}, {(b, ai); i ∈ ω}) , [2]P.A.AbdullaandA.Nyl´en. Timed Petri nets and BQOs. In Applications and theory of Petri nets 2001, we have dim PF (X) = 1 but dim X = ∞. vol. 2075 of Lecture Notes in Computer Science, pp. 53– 70. Springer, Berlin, 2001.

Lemma 10 1. L1 has NIA. [3] Y. Akama. “Set systems: Order types, continuous non- 2. A quasi-ordering (L1, ⊇) is order-isomorphic to deterministic deformations, and quasi-orders,” Theor. PF ( (L1, ⊇) ).TheyareWQOs. Comput. Sci., vol. 412, no. 45, pp. 6235–6251, 2011. 3. None of (L1, ⊇) and PF ( (L1, ⊇) ) is a BQO. [4] B. Berard. “Literal shufﬂe,” Theor. Comput. Sci., 4. None of L1 and |PF ( (L1, ⊇) )| is a BESS. vol. 51, pp. 281–299, 1987. <ω <ω 5. (L1 ) is not an FESS,but|PF ( (L1, ⊇) )| is. [5] B. Bollob´as. Combinatorics. Cambridge University Press, Cambridge, 1986. Set systems, hypergraphs, Proof. (1) By [10 Proposition 3.3]). families of vectors and combinatorial probability. 03

16 Progress in Informatics, No. 9, pp.9–18, (2012)

[6] D. P. Bovet and S. Varricchio. “On the regularity of [23] J. B. Kruskal. “Well-quasi-ordering, the Tree Theorem, languages on a binary alphabet generated by copying and Vazsonyi’s conjecture,” Trans. Amer. Math. Soc., systems,” Inform. Process. Lett., vol. 44, no. 3, pp. 119– vol. 95, pp. 210–225, 1960. 123, 1992. [24] D. Kühn. “On well-quasi-ordering infinite trees—Nash- [7] F. D’Alessandro and S. Varricchio. “Well quasi-orders Williams’s theorem revisited,” Math. Proc. Cambridge in formal language theory,” In M. Ito and M. Toyama, Philos. Soc., vol. 130, no. 3, pp. 401–408, 2001. Ed., Developments in Language Theory, vol. 5257 of [25] M. Kummer and F. Stephan. “Weakly semirecursive sets Lecture Notes in Computer Science, pp. 84–95. Springer and r.e. orderings,” Ann. Pure Appl. Logic, vol. 60, no. 2, Berlin / Heidelberg, 2008. pp. 133–150, 1993. [8] M. de Brecht, 2012. Private communication. [26] S. Lange, T. Zeugmann, and S. Zilles. “Learning in- [9] M. de Brecht. Topological and Algebraic Aspects of dexed families of recursive languages from positive Algorithmic Learning Theory. PhD thesis, Graduate data: A survey,” Theor. Comput. Sci., vol. 397, no. 1-3, School of Informatics, Kyoto University, 2009. pp. 194–232, 2008. Forty Years of Inductive Inference: [10] M. de Brecht and A. Yamamoto. “Mind change com- Dedicated to the 60th Birthday of Rolf Wiehagen. plexity of inferring unbounded unions of restricted pat- [27] A. Marcone. “Foundations of BQO theory,” Trans. tern languages from positive data,” Theor. Comput. Sci., Amer. Math. Soc., vol. 345, no. 2, pp. 641–660, 1994. vol. 411, no. 7-9, pp. 976–985, 2010. [28] A. Marcone. “Fine analysis of the quasi-orderings on [11] P. de Groote, B. Guillaume, and S. Salvati. “Vector ad- the power set,” Order, vol. 18, no. 4, pp. 339–347 dition tree automata,” In 19th Annual IEEE Symposium (2002), 2001. on Logic in Computer Science, pp. 64–73, Washington, [29] T. Motoki, T. Shinohara, and K. Wright. “The correct DC, USA, 2004. IEEE Comput. Soc. definition of finite elasticity: corrigendum to identifi- [12] D. H. J. de Jongh and R. Parikh. “Well-partial orderings cation of unions,” In COLT ’91: Proceedings of the and hierarchies,” Nederl. Akad. Wetensch. Proc. Ser. A fourth annual workshop on Computational learning the- 80=Indag. Math., vol. 39, no. 3, pp. 195–207, 1977. ory, p. 375, San Francisco, CA, USA, Morgan Kauf- [13] R. Diestel. Graph theory, vol. 173 of Graduate Texts mann Publishers Inc., 1991. in Mathematics. Springer, Heidelberg, fourth edition, [30] C. St. J. A. Nash-Williams. On well-quasi-ordering 2010. infinite trees. Proc. Cambridge Philos. Soc., vol. 61, [14] A. Ehrenfeucht, D. Haussler, and G. Rozenberg. “On pp. 697–720, 1965. regularity of context-free languages,” Theor. Comput. [31] C. St. J. A. Nash-Williams. “On better-quasi-ordering Sci., vol. 27, no. 3, pp. 311–332, 1983. transfinite sequences,” Proc. Cambridge Philos. Soc., Ram- [15] R. L. Graham, B. L. Rothschild, and J. H. Spencer. vol. 64, pp. 273–290, 1968. sey theory. Wiley-Interscience Series in Discrete Math- [32] M. Ogawa. “Well-quasi-orders and regular ω- ematics. John Wiley & Sons, Inc., New York, 2nd edi- languages,” Theor. Comput. Sci., vol. 324, no. 1, pp. 55– tion, 1980. A Wiley-Interscience Publication. 60, 2004. [16] G. Higman. “Ordering by divisibility in abstract algebras,” Proc. London Math. Soc. (3), vol. 2, pp. 326–336, [33] R. Rado. “Partial well-ordering of sets of vectors,” 1952. Mathematika, vol. 1, pp. 89–95, 1954. [17] M. Ito. Algebraic theory of automata and languages. [34] C. Reutenauer. Free Lie algebras, vol. 7 of London River Edge, NJ, World Scientific Publishing Co. Inc., Mathematical Society Monographs. New Series.New 2004. York, The Clarendon Press Oxford University Press, [18] P. Janˇcar. “A note on well quasi-orderings for power- 1993. Oxford Science Publications. sets,” Inform. Process. Lett., vol. 72, no. 5-6, pp. 155– [35] N. Robertson and P. Seymour. “Graph minors XXIII. 160, 1999. Nash-Williams’ immersion conjecture,” J. Combin. [19] C. G. Jockusch, Jr. “Semirecursive sets and positive re- Theory Ser. B, vol. 100, no. 2, pp. 181–205, 2010. ducibility,” Trans. Amer. Math. Soc., vol. 131, pp. 420– [36] G. Rozenberg and A. Salomaa, Ed. Handbook of formal 436, 1968. languages, vol. 3: beyond words, New York, NY, USA, [20] C. G. Jockusch, Jr. and J. C. Owings, Jr. “Weakly Springer-Verlag New York, Inc., 1997. semirecursive sets,” J. Symbolic Logic, vol. 55, no. 2, [37] J. Sakarovitch. Elements of automata theory.Cam- pp. 637–644, 1990. bridge University Press, Cambridge, 2009. Translated [21] S. Jukna. Extremal combinatorics. Texts in Theoretical from the 2003 French original by Reuben Thomas. Computer Science. An EATCS Series. Springer-Verlag, [38] T. Shinohara and H. Arimura. “Inductive inference of Berlin, 2001. With applications in computer science. unbounded unions of pattern languages from positive [22] M. Kanazawa. Learnable Classes of Categorial Gram- data,” Theor. Comput. Sci., vol. 241, pp. 191–209, 2000. mars. Studies in Logic, Language and Information. [39] K. Wright. Identification of unions of languages drawn Stanford, CA, CSLI Publications, 1998. from an identifiable class. In COLT ’89: Proceedings of 03

A new order theory of set systems and better quasi-orderings 17

the second annual workshop on Computational learning the class of sets positively reducible [19] to some sets in theory, pp. 328–333, San Francisco, CA, USA, 1989. L “uniformly” via a single recursive function Morgan Kaufmann Publishers Inc. i i∈v 2 fR(x) = 2 . Appendix R(x,v) 5 Ramsey’s numbers for well-founded According to [19], the class of semirecursive sets is trees and order type of set systems closed by the positive reduction (equivalent to an effec- Let Xi be a quasi-ordering with the maximal order tive continuous deformation, in spirit), and a semirecur- type otp(Xi) <ωand Li be a set system with the order sive set is exactly an initial segment of some recursive ω type dim Li <ω(i = 1, 2). Let Ra(n, m)betheRamsey linear ordering on . number [15] of n and m.Thenwehave Deﬁnition 10 AsetM ⊆ ω is called semirecur- 1. [3 Lemma 6] For the memberwise union L ∪ 1 sive [19] if there is a recursive function ψ of two vari- L = {L ∪ L ; L ∈L , L ∈ L }, 2 1 2 1 1 2 2 ables such that dim(L ∪L) + 1 < Ra(dim L + 2, dim L + 2). 1 2 1 2 (x ∈ M ∧ y M) ∨ (x M ∧ y ∈ M)

2. [3 Theorem 8] otp (X1 ∩X2) < Ra(otp(X1) + =⇒ ψ(x, y) ∈{x, y}∩M. (5) 1, otp(X2) + 1). We wish to generalize these two for the case dim L In [20], Jockusch and Owings introduced a following i ⊆ ω (i = 1, 2) being general ordinal numbers. To directly generalization of a semirecursive set: M is semi- generalize the proof argument of the two, we pose a r.e. if and only if there exists a partial recursive function ψ , ∈ ω following question. By a tree, we mean a prefix-closed of two variables such that for all x y set of possibly infinite sequences. A well-founded tree x ∈ M ∨ y ∈ M =⇒ ψ(x, y) ∈{x, y}∩M . is, by definition, a tree with all the elements being finite sequences. Furthermore, they introduced a following generalization of a semi-r.e. set: M is weakly semirecursive if and Conjecture 3 Is there a reasonably simple, ordinal bi- only if there exists a partial recursive function ψ of two nary (partial) function F on ordinal numbers such that variables such that the condition (5) holds. “for all ordinal numbers β and γ there exists an ordi- ψ α ≤ β, γ The (partial) function is called a selector function nal number F( ) with a following property: for of the semirecurisve (semi-r.e., weakly semirecursive) any coloring of any well-founded tree T0 of order type set M. α with red and black, either there is a well-founded tree T1 of order type β such that T1 is homeomorphically We adapt the notion of the initial segments of partial embedded into the red nodes of T0, or there is a well- orderings [25 p. 136], as follows: founded tree T2 of order type γ such that T2 is homeo- ω morphically embedded into the black nodes of T0.” Definition 11 For any quasi-ordering on ,wesay M ⊆ ω is an initial segment of , if and only if any of 6 Initial segments of quasi-ordering : M is strictly smaller with respect to than any of the computability theoretic view complement M. For every nonnegative integer z,aset{z ,...,z } of 1 m Every initial segment of a quasi-ordering is trivial, if nonnegative integers z ,...,z with z = 2z1 + ···+ 2zm 1 m and only if the undirected graph induced by the quasi- is denoted by E . z ordering is not connected. A non-trivial initial segment Definition 9 AsetA⊆ ω is called positively reducible may have downward branching. via a recursive function f : ω → ω to B ⊆ ω (A ≤p B We characterize a weakly semirecursive sets and via f , in symbol), provided that for all x, x ∈ A if and semi-r.e. sets by initial segments of quasi-orderings. only if there exists y ∈ E such that E ⊆ B. Intu- f (x) y Theorem 4 A set M is weakly semirecursive if and itively, a finite set E means a conjunction of Boolean y only if M is an initial segment of an r.e. quasi-ordering. variables, and a finite set E f (x) means a disjunction of such conjunctions Ey over y ∈ E f (x). We write A ≤p B Proof. (⇒) By [25 Theorem 4.1]. (⇐) Let the witness- if there exists a recursive function f : ω → ω such that ing quasi-ordering be ≤.Put A ≤ Bviaf. ⎧ p ⎪ , ≤ ⎨⎪ x (x y and x y); We observe that for any recursive relation R ⊆ ω × ψ(x, y):= ⎪ y, (y ≤ x and x y); (6) <ω ⎩⎪ [ω] and for any class L⊆P(ω), the image OR[L]is ↑, otherwise. 03

18 Progress in Informatics, No. 9, pp.9–18, (2012)

Then ψ is clearly a partial recursive function. Assume 7 A new order type of a set system x ∈ M y. Because M is an initial segment of ≤ in a Definition 12 A learning sequence of a set system L⊆ sense of Definition 11, we have x ≤ y and x y.Bythe P(T) is, by definition, a possibly infinite sequence definition of ψ,wehaveψ(x, y) = x. On the other hand, assume x M y.Theny ≤ x and x y. To sum up, t0, A1, t1, A2,... , tn, An+1 ψ(x, y) ∈{x, y}∩M. Thus M is a weakly semirecursive < { ,..., }⊆ ∈L set with ψ being a selector function. Q.E.D. such that for each i n t0 ti Ai+1 .In particular, we call the sequence bad if Ai+1 ti+1 for We can prove a similar result for semi-r.e. sets. each i. We say a set system L⊆P(T) has infinite elasticity, Theorem 5 A set M is semi-r.e. if and only if M is a lin- provided that there are infinite bad learning sequences. early ordered initial segment of an r.e. quasi-ordering. Otherwise, we say L has an FE, and call L an FESS. Let T be a well-founded tree. For each node σ of T, Proof. Only if-part is by [25 Theorem 5.1]. To prove let the ordinal number |σ| be the supremum of |σ | + 1 ∈ the converse, assume x M without loss of generality. such that σ ∈ T is an immediate extension of σ.Then ∈ ≤ ≤ When y M,wehavex y or y x because M is the order type |T| of the well-founded tree T is defined ψ , ∈{ , }∩ linearly ordered. By (6), we have (x y) x y M. by the ordinal number | | assigned to the root of T. ≤ When y M, x y and x y because M is an initial For a tree T which is not well-founded, let |T| be ∞. ≤ ψ , = segment of .By(6),wehave (x y) x. Q.E.D. The order type of L, denoted by dim L, is, by definition, the order type of the tree of bad learning se- ≤ A lemma similar to “If A p B and B is semirecur- quences of L. sive, then A is semirecursive” [19 Theorem 4.2] holds

for semi-r.e. sets and weakly semirecursive sets. In the premise of Proposition 1, we cannot replace the domain of the continuous function O : {0, 1} M → Lemma 11 If A ≤ M and M is semi-r.e. (weakly p L with a set system M. We have following counterex- semirecursive, resp.), then so is A. ample: M = {{i} ; i ∈ ω} is a discrete subspace of the { , }ω Proof. Let ψ be a selector function of M. Because product topology 0 1 and hence any function from M L A ≤ M,thesetA is many-one reducible to M,by[19 the relative topology to a set system is continuous p L Theorem 4.2 (ii)]. So there exists a recursive function g even if is not an FESS. such that Yohji AKAMA ∈ ⇐⇒ ∈ . x A g(x) M (7) Yohji AKAMA obtained a PhD in Deﬁne a partial recursive function ψ by Sciencce from the Tokyo Institute of ⎧ Technology in 1995 for work on logic ⎪ , ψ , = ⎨⎪ x ( (g(x) g(y)) g(x)); in computer science. He is currently ψ (x, y) = ⎪ y, (ψ(g(x), g(y)) = g(y)); (8) ⎩⎪ an associate professor of Mathemati- ↑, (otherwise). cal Institute, Graduate School of Sci- (i) Assume M is weakly semirecursive. Suppose x ∈ ence, Tohoku University. His current research inter- A y without loss of generality. By (7), g(x) ∈ sts and efforts are focused around the study of com- M g(y). Thus ψ(g(x), g(y)) ∈{g(x), g(y)}∩M.By binatorial, discrete geometric aspects of computational g(y) M,wehaveψ(g(x), g(y)) = g(x) ∈ M. Hence and statistical learning theory, chemistry, physics, ψ (x, y) = x ∈{x, y}∩A. Therefore A is weakly semire- and so on. cursive with a selector function ψ . (ii) Assume M is semi-r.e. Suppose x ∈ A or y ∈ A. Then g(x) ∈ M or g(y) ∈ M.Soψ(g(x), g(y)) ∈ {g(x), g(y)}∩M.Whenψ(g(x), g(y)) = g(x), by a simi- larargumentof(i),wehaveψ (x, y) ∈{x, y}∩A.When ψ(g(x), g(y)) = g(y), ψ (x, y) = y ∈{x, y}∩A. Thus A is semi-r.e. with a selector function ψ . Q.E.D.

Corollary 4 Let R ⊆ ω×[ω]<ω be a recursive relation. Then if A is semirecursive (semi-r.e., weakly semirecursive resp.), then so is B ⊆ ω where B = OR(A).