IST 4 Information and Logic mon tue wed thr fri sun 3 M1 oh T = today 1 10 oh M1 oh oh x= hw#x out 17 oh 1 2 oh M2 x= hw#x due 24 oh oh 2 1 oh M2 oh midterms oh = office hours 8 3 oh oh oh Mx= MQx out 15 oh 3 4 oh 22 oh oh T CP1 Mx= MQx due 4 5 CP 29 oh oh oh CP = challenge problem CP2 5 oh 5 oh Last Lecture: languages for biology - Stochastic chemical networks Stochastic logic design The B- algorithm Duality - Molecular switches DNA strand displacement 1 1 - Stochastic flow networks 2 2 Feedback helps! 1 2 1 2 3 2 1 3 A relay circuit is a physical system for syntax manipulation Relay circuits are not the only option! AND gate OR gate Linear Threshold (LT) gate (deep) learning: adjusting the weights a t b Circuits with Gates functional gates – syntax boxes AON: AND, OR, Not LT: Linear Threshold Questions about building blocks? Feasibility Given a set of building blocks: What can/cannot be constructed? Efficiency and complexity If feasible, how many blocks are needed? Algorizm? AND, OR and NOT (AON) What is the function computed by this circuit? a a b 3 total number of gates in the circuit a 2 b longest path from input to output – counting the b number of gates Every 0-1 Boolean Function Can be Implemented Using A Depth Two AON Circuit Implement the DNF representation: OR of many ANDs XOR of 3 Variables abc XOR(a,b,c) 000 0 001 1 010 1 011 0 100 1 101 0 110 0 111 1 XOR of 3 Variables Depth = 2 a b c > Size = 5 a b > c > a b c > is the complement a b c > XOR of More Variables? How many gates in a depth-2 circuit for XOR of n variables with AON? Surprisingly, this is the optimal size for depth-2 Depth-2 AON Circuit for XOR Theorem: An optimal size depth-2 AON circuit for has gates Proof: The construction follows from the DNF representation: a b normal terms + one OR gate c > a b > The lower bound: c > WLOG a b c > a (i) Every AND gate must have b ??? all n inputs c > (ii) Every AND gate computes a normal term DNF is a representation, hence, Without Loss Of there are AND gates Generality?? Depth-2 AON Circuit for XOR Theorem: An optimal size depth-2 AON circuit for has gates Proof (cont): Need to prove: (i) Every AND gate must have all n inputs By contradiction: Assume that there is a gate G with n-1 Making G=1 ? inputs . Say x1 is missing from G Assume that: set a variable to 1 and a complement to 0 0 a a 1 b 1 b > > c 0 c a b > c > a b Hence, the output of the circuit is 1 c > a b OR gate has input of 1 c > Depth-2 AON Circuit for XOR Theorem: An optimal size depth-2 AON circuit for has gates Proof (cont): Assume that: So what? Hence, the output of the circuit is 1 (OR gate has input of 1) Note that the following two assignments force the output of the circuit to be 1: Those assignments have different parities Q Contradiction!! How many gates in a depth 2 circuit for XOR of n variables with AON? It is optimal size for depth-2 n=4, depth 2, size 9 Q: for n=4, arbitrary depth, suggest a circuit for XOR with size less than 9? Size 8 AON Circuit for XOR of Four Variables size 5 size 3 a b XOR(x,y,z) XOR(x,y) XOR(a,b,c,d) c d Arbitrary depth circuit for XOR of n variables with AON? Idea: Compute a large XOR by using a circuit of small XOR gates AON Circuit for XOR Idea: Compute a large XOR by using a circuit of small XOR gates 8 variables in-degree = 2 Tree leaf = input edge edge = wire node = XOR gate XOR Q: Can we do better for 8 variables? Idea: Compute a large XOR by using a circuit of small XOR gates 8 variables Circuit size in AON gates? Size = Node size X number of nodes Note that we need size 129 in depth-2… 3 X 7 = 21 XOR Q: Can we do better for 8 variables? Idea: Use a larger in-degree? Size 18 for 8 variables 9 variables Size = Node size X number of nodes 5 X 4 = 20 Note that we need size 21 with in-degree 2 XOR In general, we can prove that degree-3 XOR trees are the best! Size is Idea: Use a larger in-degree? Size 18 for 8 variables 9 variables Size = Node size X number of nodes 5 X 4 = 20 Note that we need size 21 with in-degree 2 XOR AON Constructions for XOR n=4 circuit kind size 9 AON, d-2 optimal 8 AON lower bound: not optimal AON Circuit for XOR We have a construction of size we know how to prove a lower bound of 2n-1 2 3 3 3 5 5 4 7 8 5 9 10 6 11 13 7 13 15 8 15 18 AON Circuit for XOR We have a construction of size we know how to prove a lower bound of 2n-1 2 3 3 Matt Cook proved (2005) 3 5 5 next gap that an AON circuit of 4 8 8 size 7 for XOR does 5 10 10 not exist he used 6 12 13 a computer search 7 14 15 8 16 18 A circuit for n=6 with 12 AND gates (7n – 4)/3 2 3 3 A recent result by 3 5 5 Kombarov, 2015 4 8 8 next gap An upper bound! 5 10 10 6 12 12 7 14 15 8 16 17 Ingo Wegener, 1991 (will be posted on the class web site) “The complexity of the parity function in unbounded fan-in, unbounded depth circuits.” Prove a matching upper/lower bounds and get an MSc in CS New upper bound: (7n-4)/3 The problem: Most functions require a large circuit size - in the number of inputs 4x(3-1)=8 The circuit complexity problem: While most functions require a large circuit size - in the number of inputs Currently we can only prove lower bounds... Show a function that requires circuit size! Size: total number of gates in the circuit Circuits with Gates LT: Linear Threshold LT: Linear Threshold Neuron – Neural Gate LT: Linear Threshold What is the function computed by this gate? 1 -2 0 0 -2 0 1 0 1 -1 0 1 0 -1 0 1 1 0 1 Neural Circuits feasibility LT: Linear Threshold Q: Are LT gates magical? 2 input Linear Threshold (LT) gate LT: Linear Threshold Q: Are LT gates magical? Idea: A Linear Threshold is Magical Can compute AND, OR and NOT We showed that we can compute the AND function with an LT gate 1 -2 0 0 -2 0 1 0 1 -1 0 1 0 -1 0 1 1 0 1 Can We Compute an OR Function with an LT Gate? 1 -1 0 0 -1 0 1 0 1 0 1 1 0 0 1 1 1 1 1 Can We Compute a NOT with an LT Gate? -2 1 Can we compute NOT without sgn? More Variables for AND? Hence is an AND More Variables for OR? Hence is an OR Circuits Efficiency and complexity The Functions of the Adder d1 d2 c 2 symbol adder c s sum carry d1 d2 c 2 symbol adder c XOR with a Single LT Gate s Is it possible to compute with a single LT gate? Idea: Find weights w0, w1 and w2 such that: d1 d2 c 2 symbol adder c XOR with a Single LT Gate s Is it possible to compute with a single LT gate? Answer : NO Proof: By contradiction assume it is possible and reach a contradiction Q d1 d2 c 2 symbol adder c XOR with More Variables? s Is it possible to compute Need LT circuits with a single LT gate? for XOR! Idea: suppose that it is possible, and reach a contradiction However, And, Contradiction d1 d2 c 2 symbol adder c MAJ with a Single LT Gate s Is it possible to compute with a single LT gate? |X| MAJ 0 0 1 0 2 1 3 1 AND, OR, XOR and MAJ are symmetric functions Q: Which symmetric functions are in LT1? |X| AND OR XOR MAJ 0 0 0 0 0 1 0 1 1 0 2 0 1 0 1 3 1 1 1 1 LT1 LT1 not LT1 LT1 LT1 = the class of Boolean functions that can be realized by a single LT gate. Definition: A symmetric Boolean function is in TH if it has at most a single transition in the symmetric function table = a transition |X| AND OR XOR MAJ 0 0 0 0 0 1 0 1 1 0 2 0 1 0 1 3 1 1 1 1 In TH Not in TH The Class TH The Class TH - Single Transition = a transition Q: what is |TH| ? A: 2n+2 the number TH functions... |X| TH0 TH1 TH2 TH3 TH0 TH1 TH2 TH3 0 1 0 0 0 0 1 1 1 1 1 1 0 0 0 0 1 1 2 1 1 1 0 0 0 0 1 3 1 1 1 1 0 0 0 0 Claim: Proof: 0 1 Q The Class TH is in LT1 |X| TH0 TH1 TH2 TH3 TH0 TH1 TH2 TH3 0 1 0 0 0 0 1 1 1 1 1 1 0 0 0 0 1 1 2 1 1 1 0 0 0 0 1 3 1 1 1 1 0 0 0 0 Need LT circuits for XOR! AON and Linear Threshold Circuits XOR example XOR of Three Variables Size 5 is optimal for AON depth 2 Depth = 2 a b c > Size = 5 a b > c > a b c > is the complement a b c > LT gates are MORE Powerful FOR XOR: Size 5 is optimal for AON depth 2 Size 4 LT depth 2 1 1 -1 1 -1 1 -1 1 1 -2 -1 1 1 1 -3 1 LT gates are MORE Powerful TH functions LT-l = LT layered A+B+C -2+A+B+C inputs go to first layer only A B C 0 0 1 0 1 -1 1 A 1 1 1 0 2 0 1 -1 2 1 0 0 1 -1 1 3 1 0 1 2 0 -1 B 1 -1 1 1 -2 -1 1 1 C 1 -3 1 Can take the sgn or add 1 XOR Function: Size of LT vs AON in Depth 2 AON 5 * LT-l 4 * * = it is optimal Exponential gap in size AON 5 LT-l 4 General construction for symmetric functions Linear Threshold Circuits symmetric functions LT Depth-2 Circuits TH1 ??? -1 + TH2 |X| TH1 TH2 TH1+TH2-1 0 0 1 0 1 1 1 1 2 1 0 0 Generalization |X| f(x) 0 0 1 1 2 1 3 0 4 0 Generalization |X| f(x) 0 0 1 1 2 1 3 0 4 0 Generalization |X| f(x) TH1 0 0 0 1 1 1 2 1 1 3 0 1 4 0 1 Generalization |X| f(x) TH1 TH3 0 0 0 1 1 1 1 1 2 1 1 1 3 0 1 0 4 0 1 0 Generalization |X| f(x) TH1 TH3 Σ -1 0 0 0 1 0 1 1 1 1 1 2 1 1 1 1 3 0 1 0 0 4 0 1 0 0 |X| f(x) TH1 TH3 Σ -1 0 0 0 1 0 1 1 1 1 1 2 1 1 1 1 3 0 1 0 0 4 0 1 0 0 Generalization to SYM -1 + Q: What is the generalization to arbitrary symmetric functions? Generalization to SYM Q: What is the generalization to arbitrary symmetric functions? A: Consider the symmetric function table, it is a sum of non-overlapping 1-intervals 0 1 0 1 Sum of two TH functions Back to XOR n TH gates for XOR of n variables 0 0 1 1 2 0 3 1 4 0 5 1 LT-l Circuit Design Algorithm for SYM f(X) 0 1 1 1 2 0 3 1 Subtract 1 for every 4 1 isolated 1-block 5 0 6 1 7 1 The Layered Construction for SYM Some History Saburo Muroga 1925- 2009 1959 Was born in Japan Majority Decision PhD in 1958 from Tokyo U, Japan 1960-1964: Researcher at IBM Research, NY 1964-2002: professor at the University of Illinois, Urbana-Champaign (4,2,2,3) (6,2,0,2) LT1 = Can be computed by a single LT gate In LT1 – show a construction Not in LT1 – show a proof .