The Nonnegative Rank of a Matrix: Hard Problems, Easy Solutions

arXiv:1605.04000v2 [math.CO] 28 Dec 2017 otisa xlcteapeo 21 a of example explicit an contains ale ae 1]cnan eae eutta sbt oegenera more both is that result p related the a of contains solution [18] another paper arXiv with on earlier concurrently which uploaded appears problem, problem been Cohen–Rothblum Cohen–Rothblum has the which the rat of 19 of solution of solution (Our sum a a now. as gives not This but matrices matrices. rank-one nonnegative 19 of sum ttsis hyas ieaprilslto fti rbe.W oet note We for asked problem. but this problem of similar solution a prob partial of this solution a consider a give [13] contains also factoriza Sturmfels rational they and optimal Robeva, statistics; of Kubjas, lack possible [16]. theo the theory the and Vavasis is [23], ranks polytopes nonnegative literature. of of the application co in notable algorithmic Another discussed and rank. widely problem Cohen–Rothblum been the has between connection it and ago, years arcsta u to sum that matrices the to related ssae ndffrn em,bti sesnilyteods reference oldest the essentially is Ravikuma of. it aware and completed, Jiang are but by we be terms, [11] rank to different paper nonnegative the about in in was stated sho contained is writing is is proof present this similar we p When a that the proof knowledge. and The b special mathematics, [21] considerations. any applied paper geometric in The ingenious result compute. on this to to NP-hard ac reference these is standard of rank one nonnegative and the matrices, that nonnegative on problems known widely [5], inclu mechanics science, quantum [13], ran modern nonnegative statistics of given [23], branches of optimization different combinatorial matrix in a with applications matrix important given a of approximation A k sartoa arxwt ongtv rank nonnegative with matrix rational a is o hc hr exist there which for hspprpeet obntra prahta ed ovr s very to leads that approach combinatorial a presents paper This nte eutta eoti saslto fthe of solution a is obtain we that result Another Let 2010 e od n phrases. and words Key A ahmtc ujc Classification. Subject Mathematics a oe n19 n eandopen. nonneg remained V rational and of on 1993 work in problem and the Cohen–Rothblum posed Jiang before was the years by of fifteen paper solution matr terms the a a different in of in contained topic rank essentially this nonnegative is the proof that First This stating matrices. Vavasis nonnegative by on result problems known widely two of Abstract. eamti ihnneaiera nre.The entries. real nonnegative with matrix a be ongtv arxfactorization matrix nonnegative H ONGTV AKO MATRIX: A OF RANK NONNEGATIVE THE sn lmnaylna ler,w eeo ehiu tha technique a develop we algebra, linear elementary Using A ADPOLM,ES SOLUTIONS EASY PROBLEMS, HARD hspolmwspsdi h onainlppr[]mr hn20 than more [5] paper foundational the in posed was problem This ? ongtv arxfcoiain opttoa complex computational factorization, matrix Nonnegative k ongtv akoemtie htsmto sum that matrices rank-one nonnegative × AOLVSHITOV YAROSLAV 1. 1mti ihitga nre htcnb rte sa as written be can that entries integral with matrix 21 50,1A3 68Q17. 15A23, 15A03, Introduction k oteeexist there do , 1 rbe,wihi h akt n h optimal the find to task the is which problem, oe–ohlmproblem Cohen–Rothblum oiiesemidefinite positive ongtv rank nonnegative egv hr ro fthe of proof short a give we , vss eody epresent we Secondly, avasis. k xi Phr ocompute. to NP-hard is ix aiua,wodiscussed who Ravikumar, tv atrztos which factorizations, ative rational yo xeddformulations extended of ry oa ongtv rank-one nonnegative ional n oeseic There specific: more and l e ncneto modern of context in lem in sadffiut nthis in difficulty a is tions olm e 4.As,the Also, [4]. see roblem, opihet saproof a is complishments ofi otisi based is contains it roof ed osolutions to leads t tadde o require not does and rt peiyo nonnegative of mplexity aai a eoea become has Vavasis y oteN-adesof NP-hardness the to [14], mining data ding stemnsrp [19], manuscript the as 2]dmntae the demonstrates [21] otsltoso two of solutions hort n ayothers. many and ongtv rank-one nonnegative .Ti rbe has problem This k. A ity. eppr[] which [7], paper he a enoe until open been has .TerLma3.3 Lemma Their r. of h uhrlearned author the hscnetis concept This . ak u paper Our rank. A suethat Assume . stesmallest the is 2 YAROSLAV SHITOV is a real matrix A of conventional rank five which admits a nonnegative rank factorization but no such factorization is rational in the entries of A.)

2. Our technique Let A be a nonnegative matrix with entries in a subﬁeld F R. The nonnegative rank of A ⊂ with respect to F (denoted rankF+ A) is the smallest k for which A is a sum of k nonnegative rank-one matrices over F. In particular, the quantity rankR+(A) is the usual nonnegative rank as deﬁned in the introduction. Combinatorial methods of studying the nonnegative rank have a long history, see the recent survey [9], the older papers [8, 15], and references therein. Methods that employ zero patterns of matrices are still developing and being used in modern investigations [2, 10]. As we will see later, the combinatorial analysis of zero patterns plays a crucial role in our approach. Let α ,...,αn be nonnegative real numbers. The 5 (n + 4) matrix 1 ×

α1 ... αn 1 11 1  1 ... 1 1100 (α ,...,αn)= 0 ... 0 0110 B 1    0 ... 0 0011    0 ... 0 1001   will be particularly important in our consideration. Its lower-right 4 4 submatrix × 1 1 0 0 0 1 1 0 = B0 0 0 1 1   1 0 0 1   appears in the note [20], which is one of the oldest references on the topic of nonnegative matrix factorizations. The reason why 0 is so important is that it has different conventional and nonnegative ranks, — one has rankB = 3 and rank = 4. In particular, the row B0 + B0 of ones can be represented as a linear combination of the rows of 0 in different ways: one can take the sum of the first and third rows, or the sum of the secoB nd and fourth rows, or any convex combination of these two sums. This leads us to the following easy but important observation.

Observation 1. RankF (α ,...,αn)=4 if and only if α = ... = αn [0, 1] F. + B 1 1 ∈ ∩ In other words, this matrix has the following interesting property. For any ε (0, 1), the 5 5 matrices obtained from (ε) by a small perturbation of the (1, 1) entry have∈ the same× nonnegative rank as the (1, 1)B cofactor of (ε). Basic results of linear algebra show that such a property cannot hold for a conventionalB rank function of matrices over a ﬁeld. In fact, the usual rank of a matrix equals the order of the largest non-singular submatrix, so a small perturbation of an entry would lead to a matrix with rank greater than the rank of its cofactor. We use this distinction between the conventional and nonnegative rank functions in the following matrix completion problem: Given nonnegative matrices A Fm×n, B Fm×k, c F1×n and a subset I F, what is the smallest nonnegative rank of ∈ ∈ ∈ ⊂ A B (2.1) (x)= A c x...x provided that x I? Namely, we can reduce this completion version to the standard formulation of the nonnegative∈ rank problem. NONNEGATIVERANK: HARD PROBLEMS,EASYSOLUTIONS 3

Theorem 2. Let A Fm×n, B Fm×k, c F1×n be nonnegative matrices, let r, s be positive ∈ ∈ ∈ integers. We assume that either rankF+ A > r or k =1. We define 0 00 0 . . . .  A B . . . .   0 00 0     c s...s 1 11 1  =   G  0 ... 0 1 ... 1 1 1 0 0     0 ... 0 0 ... 0 0 1 1 0     0 ... 0 0 ... 0 0 0 1 1     0 ... 0 0 ... 0 1 0 0 1    and (x) as in (2.1). Then the inequality rankF 6 r +4 holds if and only if there is a A + G ξ [s 1,s] F such that rankF (ξ) 6 r. ∈ − ∩ + A Proof. Note that is the sum of (ξ) and (s ξ,...,s ξ) up to adding zero rows and G A B − − columns to the latter matrices. Since rankF+ (s ξ,...,s ξ) = 4 by Observation 1, the ’if’ direction follows immediately. B − − Let us prove the ’only if’ direction. First, we note that the inequalities A (2.2) rankF (A B) > r and rankF > r + | + c hold trivially if rankF+ A > r. (We note that the block matrices involved in (2.2) have sizes m (n + k) and (m + 1) n, respectively.) If k = 1, then (x) can be obtained by adding a row× (a column, respectively)× to the matrix in the left (right,A respectively) inequality of (2.2). Therefore, if (2.2) were not true, we would get rankF+ (x) 6 r and complete the proof. So we can assume that the inequalities (2.2) are true. A Now we assume that G ,...,Gr are nonnegative rank-one matrices that sum to . If 1 +4 G there were a Gi with two non-zero red entries, then would have a 2 2 submatrix with G × positive entries two of which are red, — but this is not the case. Therefore, any Gi contains at most one non-zero red entry, so there are at least four Gi’s with non-zero red entries. We call these Gi’s red, denote their sum by R, and observe that they have zero black entries. Using the first inequality in (2.2), we get that there are r non-red Gi’s, and every of them has a non-zero entry either in the block corresponding to A or in the block corresponding to B. Using the second inequality in (2.2), we see that every non-red Gi contains a non-zero entry either in the block corresponding to A or in the block corresponding to c. The two previous sentences allow us to conclude that the non-red Gi’s do not contribute to the green entries of . Taking into account Observation 1, we get that, for some a [0, 1], the matrix R equals (a,...,aG ) up to the zeroes at black positions. Therefore, the top-left∈ (m + 1) (n + k) submatrixB of R is the matrix (s a) as in (2.1), which completes the proof. × G− A − Corollary 3. Let (x), be as in Theorem 2. If k =1, then we have A G rankF = min rankF (ξ)+4, + G + A where the minimum is taken over all ξ [s 1,s] F. ∈ − ∩ 3. On the complexity of nonnegative rank Corollary 3 allows us to construct a polynomial time reduction to nonnegative rank from the following problem, which is known (see [22]) as the nonnegative rank of a partial 0-1 matrix. Given a matrix X whose (i, j) entry can be 0, 1 or xij ; what is the smallest nonnegative rank of matrices that are obtained from X by assigning numbers in [0, 1] to the variables xij ? We are going to prove that this problem is NP-hard. Let G be a simple graph with vertex set V and edge set E. A clique in G is a subset U V ⊂ such that u1,u2 E for all distinct u1,u2 U. The clique covering number of G is the smallest number{ of} ∈ cliques needed to cover V .∈ Let X = X(G) be the matrix (whose rows and 4 YAROSLAV SHITOV columns are indexed with vertices of G) defined by Xvv = 1, Xuv = xuv if u, v are adjacent, and Xuv = 0 otherwise. Let U1,...,Uc be cliques whose union is V , and we assume without loss of generality that i i these cliques are disjoint. For all i 1,...,c , we define the matrix H as Hαβ = 1 if i ∈ { } α, β Ui and H = 0 otherwise. Clearly, the matrix H + ... + Hc has nonnegative rank at ∈ αβ 1 most c and can be obtained from X by replacing the variables with zeros and ones. Conversely, let M 1,...,M r be nonnegative rank-one matrices whose sum can be obtained from X by replacing the variables with numbers in [0, 1]. If one of the sets V l = v l l { ∈ V Mvv > 0 was not a clique, we would find non-adjacent distinct vertices u, v in V . This | } l 1 r would imply Xuv = 0 and contradict Muv > 0. So we see that V ,...,V are cliques, and their union is V . This proves that the clique covering number of G equals the smallest nonnegative rank of matrices that are obtained from X(G) by replacing the variables with numbers in [0, 1]. Therefore, since the clique covering number is NP-hard to compute (see [12]), so is the nonnegative rank for partial 0-1 matrices. The observation in the beginning of this section shows that the standard version of the nonnegative rank problem is NP-hard as well. As pointed out in the introduction, Lemma 3.3 in [11] contains essentially the same result. Namely, Jiang and Ravikumar prove the NP-hardness of the normal set basis problem, which is a reformulation of the more commonly known biclique partition number problem. We refer the reader to [3] for a discussion of these two formulations and further complexity results. There is also a straightforward correspondence between the biclique partition number of a bipartite graph and the nonnegative rank of its adjacency matrix, see Remark 6.4 in [15] and Lemma 2.2 in [8]. We hope that these references can help an interested reader to translate the proof of Lemma 3.3 in [11] into the language of matrix theory and come up with the NP-hardness proof for nonnegative rank.

4. Nonnegative rank depends on the field We proceed with a solution of the Cohen–Rothblum problem. To this end, we consider the matrix 2 2 21 0 000000000000 1 1 1 1 12101 0000000000000000 0 0 1 2 0 00000000 1 1 1 1 0 0 0 0   0 1 0 0 2 0 00 0 1 1 1 1 00000000   0 1 1 221111 000000000000   0 0 0 111100 000000000000   0 0 0 000110 000000000000   0 0 0 000011 000000000000   0 0 0 001001 000000000000   0 0 0 0 1 0 00 0 1 1 0 0 00000000   0 0 0 0 0 0 00 0 0 1 1 0 00000000   0 0 0 0 0 0 00 0 0 0 1 1 00000000   0 0 0 0 0 0 00 0 1 0 0 1 00000000   0 0 0 1 000000000 1 1 0 0 0 0 0 0   0 0 0 0 000000000 0 1 1 0 0 0 0 0   0 0 0 0 000000000 0 0 1 1 0 0 0 0   0 0 0 0 000000000 1 0 0 1 0 0 0 0   1 0000000000000000 1 1 0 0   0 0000000000000000 0 1 1 0   0 0000000000000000 0 0 1 1   0 0000000000000000 1 0 0 1   and denote it by . We are going to show that the rational and real nonnegative ranks of are diﬀerent. OurM strategy is to apply Theorem 2 to , and we assume that F is a ﬁeld Mas in Section 2. Let us remove the last twelve rows and columnsM from , replace the (1, 1), M NONNEGATIVERANK: HARD PROBLEMS,EASYSOLUTIONS 5

(3, 4), (4, 5) entries by the variables a,b,c, and denote the resulting matrix by 1(a,b,c). The threefold application of Corollary 3 implies M

(4.1) rankF+ = min rankF+ 1(a,b,c)+12. M a,b,c∈[1,2]∩F M Observing that the 3 3 submatrix defined as the intersection of the second, third, and fourth rows and the first three× columns of has rank three, we apply Theorem 2 with r = 3 to M 1(a,b,c). Taking into account (4.1), we get that the inequality rankF+ 6 19 holds if and Monly if there are a,b,c,d [1, 2] F such that the matrix M ∈ ∩ a 22 1 0 12101 (a,b,c,d)= 0 0 1 b 0 C   0 10 0 c   0 1 1 d d   has nonnegative rank at most three with respect to F. Basic tools of linear algebra allow us to show that the conventional rank of (a,b,c,d) exceeds three unless b = c = d = 1 √0.5 and a = 2 1/b. In particular, the rationalC nonnegative rank of cannot be less than± four, − C which implies rankQ > 20. On the other hand, for α =1+ √0.5, + M 0 0 000 0 0 α−1 1 0 √2 2 √2 0 0 − 0 α 1 0 0 1 00 0 00  1 √2 1 00 0 0 000 + 00 1 α 0 + 0 0 000       0 1 00 α 00 0 00  0 0 000       0 1 00 α 00 1 α 0  0 0 000       is a representation of (√2, α, α, α, α) as a sum of three nonnegative rank-one matrices. This C shows that rankR+ 6 19 and completes the proof that the rational and real nonnegative ranks of are different.M M 5. Note added in proof The idea of my construction comes from the earlier paper [17], which contains a similar NP-hardness proof but in the setting of Boolean matrices. As I have subsequently learned, Theorem 3.3 in [6] has the same formulation and proof as the result in [17] but in the language of graph theory. I feel sorry that I did not know about [6] when I was preparing [17] for publication, but such ocurrences seem to be ubiquitous in modern mathematics and hard to avoid. (As said above, a similar situation arose with Lemma 3.3 in [11], which states the NP-hardness of nonnegative rank but not in the terms common in applied mathematics.)

6. Acknowledgements This work was carried out at National Research University — Higher School of Economics in Moscow. I owe my gratitude to the university for their support of research activities and to my students, from whom I probably learned more than they from me. I would like to thank the editors and anonymous referees of SIAM Review for careful reading of the paper and helpful suggestions, which allowed me to improve the presentation of the results. Before I decided to include the solution of the Cohen–Rothblum problem in this paper, the manuscript [19] was undergoing the reviewing process in SIAM Journal of Applied Algebraic Geometry. Iam grateful to the referee of that journal for the detailed report and helpful comments.

References

[1] D. de Caen, D. A. Gregory, N. J. Pullman, The Boolean rank of zero-one matrices. In Proceedings of the Third Caribbean Conference on Combinatorics and Computing (Bridgetown, 1981), pages 169–173, 1981. [2] M. Conforti, G. Cornu´ejols, G. Zambelli, Extended formulations in combinatorial optimization, 4OR 8 (2010) 1–48. 6 YAROSLAV SHITOV

[3] P. Chalermsook, S. Heydrich, E. Holm, A. Karrenbauer, Nearly tight approximability results for minimum biclique cover and partition, In Proceedings of the 22nd Annual European Symposium on Algorithms (Wroclaw, 2014), pages 235–246, 2014. [4] D. Chistikov, S. Kiefer, I. Maruˇsić, M. Shirmohammadi, J. Worrell, Nonnegative Matrix Factorization Requires Irrationality, preprint (2016) arXiv:1605.06848. [5] J. E. Cohen, U. G. Rothblum, Nonnegative ranks, decompositions, and factorizations of nonnegative matrices, Linear Algebra and Its Applications 190 (1993) 149–168. [6] M. C. Golumbic, T. Hirst M. Lewenstein, Uniquely restricted matchings, Algorithmica 31 (2001) 139–154. [7] J. Gouveia, H. Fawzi, R. Z. Robinson, Rational and real positive semidefinite rank can be different, Operations Research Letters 44 (2016) 59–60. [8] D. A. Gregory, N. J. Pullman, K. F. Jones, J. R. Lundgren, Biclique coverings of regular bigraphs and minimum semiring ranks of regular matrices, Journal of Combinatorial Theory, Series B 51 (1991) 73–89. [9] S. Fiorini, V. Kaibel, K. Pashkovich, D. O. Theis, Combinatorial bounds on nonnegative rank and extended formulations, Discrete mathematics 313 (2013) 67–83. [10] S. Fiorini, S. Massar, S. Pokutta, H. R. Tiwary, R. de Wolf, Linear vs. semidefinite extended formulations: exponential separation and strong lower bounds, in Proceedings of the Forty-Fourth Annual ACM Symposium on Theory of Computing, ACM, 2012. [11] T. Jiang, B. Ravikumar, Minimal NFA problems are hard, SIAM Journal on Computing 22 (1993) 1117– 1141. [12] R. Karp, Reducibility Among Combinatorial Problems, Proceedings of the Symposium on the Complexity of Computer Computations (1972) 85–103. [13] K. Kubjas, E. Robeva, B. Sturmfels, Fixed points of the EM algorithm and nonnegative rank boundaries, Annals of Statistics 43(1) (2015), 422–461. [14] D. D. Lee, H. S. Seung, Learning the parts of objects by non-negative matrix factorization, Nature 401 (1999) 788–791. [15] J. Orlin, Contentment in graph theory: Covering graphs with cliques, Indagationes Mathematicae 80 (1977) 406–424. [16] T. Rothvoss, Some 0/1 polytopes need exponential size extended formulations, Mathematical Programming 142 (2013) 255–268. [17] Ya. Shitov, On the complexity of Boolean matrix ranks, Linear Algebra and Its Applications 439 (2013) 2500–2502. [18] Ya. Shitov, Nonnegative rank depends on the field, preprint (2015) arXiv:1505.01893. [19] Ya. Shitov, Nonnegative rank depends on the field II, preprint (2016) arXiv:1605.07173v1. [20] L. B. Thomas, Rank Factorization of Nonnegative Matrices, SIAM Review 16 (1973) 393–394. [21] S. Vavasis, On the complexity of nonnegative matrix factorization, SIAM Journal on Optimization 20 (2009) 1364–1377. [22] T. Watson, Nonnegative Rank vs. Binary Rank, Chicago Journal of Theoretical Computer Science 2 (2016) 1–13. [23] M. Yannakakis, Expressing combinatorial optimization problems by linear programs, Journal of Computer and System Sciences 43 (1991) 441–466.

Higher School of Economics, 20 Myasnitskaya Ulitsa, Moscow 101000, Russia E-mail address: [email protected]