On the Complexity of Boolean Networks Author Edgar Alberto Z´u~nigaP´erez Supervisors Dr. Edgardo Ugalde Salda~na Dr. Jes´usUr´ıasHermosillo A thesis presented for the degree of Master of Science in Physics Universidad Aut´onomade San Luis Potos´ı Posgrado en F´ısica Facultad de Ciencias San Luis Potos´ı,S.L.P., M´exico Agosto, 2019 On the Complexity of Boolean Networks Edgar Alberto Z´u~nigaP´erez Abstract The Boolean Networks are a model which has proven to be useful to model real- world systems. The Random Boolean Network model introduced by Kauffman in 1969 has been extensively used to model regulatory genetic networks and other types of systems. In this thesis, we propose the existence of a correlation between the complexity of a Boolean Network and the complexity of its constituents, i.e., the complexity of its topology and its set of updating functions. This hypothesis was tested by performing a series of experiments with the help of the implementation to approximate Kolmogorov complexity called Block Decomposition Method (BDM). First, we present a method to measure the complexity of the individual components of a Boolean Network and then, we propose a representation which can be used to measure the complexity of a Boolean Network. The results showed that this hypothesis was correct for Random Boolean Networks with small topologies given a sufficiently large set of Boolean Networks. However, it could not be generalized to larger topologies because of the enormous computational time required by the implementation of the BDM to approximate Kolmogorov complexity. Finally, the difficulties to measure the complexities of Random Boolean Networks with larger topologies inspired us to propose a novel method to measure Kolmogorov complexity. We have called this method the Block Decomposition Method with Neural Networks (BDMNN) and is based on the use of Neural Networks to perform a regression that approximates Kolmogorov complexity. These Neural Networks were trained by using random sequences for which its complexity was computed using the original BDM implementation to approximate Kolmogorov complexity. Our implementation was evaluated by performing some experiments with random sequences of bits. The results showed that our implementation is faster and requires less computational power to approximate Kolmogorov complexity than the original implementation. The only cost to be paid is a decrease in the accuracy of the results, however, we expect this error can be easily reduced with some little modifications to the method. iii Dedication I dedicate this thesis to all my teachers. v Acknowledgments I want to thank to the Universidad Aut´onomade San Luis Potos´ı,especially to the staff of its Physics Institute for their support and facilities to write this thesis. The knowledge that I acquired from my teachers in the graduate program was priceless and its dedication has paid off with this work. Finally, I also would like to thank the Consejo Nacional de Ciencia y Tecnolog´ıa(CONACYT) for their financial support throughout my studies to get the master's degree. vi Contents 1 Introduction 1 2 Review of Graph Theory 3 2.1 What is a Graph? . 3 2.2 The Vertex Degree . 5 2.2.1 Some Basic Definitions . 5 2.2.2 The Handshaking Lemma . 7 2.3 Adjacency and Incidence Matrices . 7 2.3.1 The Adjacency Matrix . 8 2.3.2 The Incidence Matrix . 8 2.4 Isomorphic Graphs . 9 2.5 Paths and Cycles . 10 2.6 Some Families of Graphs . 10 2.6.1 Null Graphs . 12 2.6.2 Complete Graphs . 12 2.6.3 Cycle Graphs . 13 2.6.4 Path Graphs . 13 2.6.5 Bipartite Graphs . 13 2.6.6 Cube Graphs . 14 2.6.7 Trees . 14 2.6.8 Circulant Graphs . 15 2.7 Counting Graphs . 15 2.7.1 Labeled Graphs . 16 2.7.2 Unlabeled Graphs . 16 2.7.3 Counting Adjacency Matrices . 16 2.8 Directed Graphs . 17 2.8.1 The Underlying Graph . 18 2.8.2 Subdigraphs . 18 2.8.3 Local Degrees in a Digraph . 18 2.8.4 Adjacency and Incidence Matrices for a Digraph . 19 2.8.5 Isomorphic Digraphs . 19 2.8.6 Counting Digraphs . 20 2.9 Random Graphs . 20 2.9.1 The Watts-Strogatz Small-World Graph Distribution . 21 2.9.2 The Barab´asi-Albert Graph Distribution . 22 2.9.3 The Uniform Graph Distribution . 23 2.10 Other Ways of Representing Graphs . 23 vii viii CONTENTS 3 Boolean Networks 25 3.1 Boolean Networks and The Random Boolean Network Model . 25 3.1.1 Boolean Networks . 25 3.1.2 Random Boolean Networks . 26 3.1.3 Topology . 26 3.1.4 Updating Functions . 28 3.1.5 Counting RBNs . 31 3.1.6 Dynamics: The State Space and Some Basic Definitions . 32 3.2 The Dynamical Phases . 35 3.3 Other Updating Schemes . 37 3.3.1 Asynchronous RBNs . 37 3.3.2 Generalized Asynchronous RBNs . 37 3.3.3 Deterministic Asynchronous RBNs . 38 3.3.4 Deterministic Generalized Asynchronous RBNs . 38 3.3.5 Mixed-context RBNs . 38 3.4 Applications of The RBNs . 38 3.5 Software to Study RBNs . 40 4 Algorithmic Complexity 41 4.1 The Kolmogorov Complexity . 41 4.1.1 The Turing Machine . 43 4.1.2 The Universal Turing Machine . 45 4.1.3 The Halting Problem . 47 4.1.4 The Formal Definition of K-Complexity . 48 4.2 Lossless Compression and Entropy as Approximations to K-Complexity 49 4.2.1 The Lossless Compression . 50 4.2.2 The Shannon Entropy . 51 4.3 Towards a Better Measurement of K-Complexity . 56 4.3.1 The Invariance Theorem . 56 4.3.2 The Algorithmic Probability and The Coding Theorem . 57 4.4 A Library to Measure The K-Complexity . 59 4.4.1 Methodology . 59 4.5 Some Applications of Kolmogorov Complexity . 61 5 The Complexity of Random Boolean Networks 63 5.1 Software and Hardware Features . 64 5.2 The Complexity of Random Sequences of Bits . 64 5.3 The Complexity of Random Graphs . 66 5.3.1 Complexity from The Watts-Strogatz Graph Distribution . 66 5.3.2 Complexity from The Barab´asi-Albert Graph Distribution . 68 5.4 The Complexity of Random Digraphs . 70 5.4.1 The Complexity from Increasing the Vertex In-Degree . 70 5.4.2 The Complexity from The Uniform Digraph Distribution . 72 5.5 The Complexity of Isomorphic Networks . 73 5.5.1 The Complexity of Isomorphic Graphs . 73 5.5.2 The Complexity of Isomorphic Digraphs . 76 5.5.3 The Complexity from The Uniform Digraph Distribution Re- visited by Considering Isomorphisms . 78 5.6 The Complexity of a Set Boolean Functions . 80 CONTENTS ix 5.6.1 The Complexity from Sets of Random Boolean Functions with Increasing Number of Inputs . 81 5.6.2 The Complexity from Sets of Random Boolean Functions with Fixed Number of Inputs . 83 5.7 The Complexity of Boolean Networks . 86 5.7.1 A Lossless Representation for Boolean Networks . 86 5.7.2 The Correlation Between the Complexity of Random Boolean Networks and The Complexity of Its Topology . 88 5.7.3 The Width of the Distribution of Complexities for Random Boolean Networks . 91 5.7.4 The Correlation Between the Complexity of Random Boolean Networks and the Complexity of its Set of Updating Functions 94 5.8 Conclusions . 97 6 A Low-cost and Faster Implementation of Kolmogorov Complexity for Binary Sequences 101 6.1 Theoretical Framework . 102 6.1.1 Machine Learning . 102 6.1.2 Neural Networks . 104 6.1.3 Machine Learning Frameworks . 120 6.2 Methodology . 120 6.2.1 Software and Hardware Features . 122 6.2.2 The Training and Target Sets . 122 6.2.3 Design of The Neural Networks . 123 6.2.4 The Learning Settings . 124 6.2.5 The Block Decomposition Method with Neural Networks . 125 6.3 Results . 126 6.3.1 The Complexity of Random Sequences of Bits . 126 6.3.2 The Error and Computational Time of the BDMNN . 127 6.3.3 The BDMNN Versus a Parallel Implementation of the BDM . 128 6.4 Conclusions . 129 7 General Conclusions 135 A Algorithms 137 A.1 Wolfram Language Algorithms . 137 A.2 Python Algorithms . 148 References 151 List of Figures 2.1 A simple graph versus a graph that is not simple. 4 2.2 Two graph diagrams that look different but represent the same graph. 5 2.3 A subgraph. 5 2.4 Examples of regular graphs. 6 2.5 Basic definitions for a graph. 7 2.6 Adjacency and incidence matrices. 9 2.7 Isomorphic graphs. 11 2.8 Example of a walk. 11 2.9 Examples of connected and disconnected graphs. 12 2.10 The first six null graphs. 12 2.11 The first six complete graphs. 12 2.12 The first six cycle graphs. 13 2.13 The first six path graphs. 13 2.14 Two examples of bipartite graphs. 14 2.15 Examples of complete bipartite graphs. 14 2.16 The first four cube graphs. 14 2.17 Two examples of tree graphs..