DOCSLIB.ORG

CSC373 Review

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
CSC373 Review CSC373 Review 373F19 - Nisarg Shah 1 BUT FIRST… Course Evaluation Low response rate 373F19 - Nisarg Shah 2 Complete your course evaluations… Check your e-mail for a link to your evaluations, or log-in to www.portal.utoronto.ca and click the Course Evals tab! 373F19 - Nisarg Shah 3 Should I expect drumrolls? Course Evaluation New response rate 373F19 - Nisarg Shah 4 Topics • Divide and conquer • Greedy algorithms • Dynamic programming • Network flow • Linear programming • Complexity • Approximation algorithms & local search • Randomized algorithms 373F19 - Nisarg Shah 5 Greedy Algorithms • Greedy algorithm outline ➢ We want to find the optimal solution maximizing some objective 푓 over a large space of feasible solutions ➢ Solution 푥 is composed of several parts (e.g. a set) ➢ Instead of directly computing 푥… o Consider one element at a time in some greedy ordering o Make a decision about that element before moving on to future elements (and without knowing what will happen for the future elements) 373F19 - Nisarg Shah 6 Greedy Algorithms • Proof of optimality outline ➢ Strategy 1: o 퐺푖 = greedy solution after 푖 steps o Show that ∀푖, there is some optimal solution 푂푃푇푖 s.t. 퐺푖 ⊆ 푂푃푇푖 • “Greedy solution is promising” o By induction o Then the final solution returned by greedy must be optimal ➢ Strategy 2: o Same as strategy 1, but more direct o Consider 푂푃푇 that matches greedy solution for as many steps as possible o If it doesn’t match in all steps, find another 푂푃푇 which matches for one more step (contradiction) 373F19 - Nisarg Shah 7 Dynamic Programming • Key steps in designing a DP algorithm ➢ “Generalize” the problem first o E.g. instead of computing edit distance between strings 푋 = 푥1, … , 푥푚 and 푌 = 푦1, … , 푦푛, we compute 퐸[푖, 푗] = edit distance between 푖-prefix of 푋 and 푗-prefix of 푌 for all (푖, 푗) o The right generalization is often obtained by looking at the structure of the “subproblem” which must be solved optimally to get an optimal solution to the overall problem ➢ Remember the difference between DP and divide-and- conquer ➢ Sometimes you can save quite a bit of space by only storing solutions to those subproblems that you need in the future 373F19 - Nisarg Shah 8 Dynamic Programming • Dynamic programming applies well to problems that have optimal substructure property ➢ Optimal solution to a problem contains (or can be computed easily given) optimal solution to subproblems. • Recall: divide-and-conquer also uses this property ➢ You can think of divide-and-conquer as a special case of dynamic programming, where the two (or more) subproblems you need to solve don’t “overlap” ➢ So there’s no need for memoization ➢ In dynamic programming, one of the subproblems may in turn require solution to the other subproblem… 373F19 - Nisarg Shah 9 Dynamic Programming • Top-Down may be preferred… ➢ …when not all sub-solutions need to be computed on some inputs ➢ …because one does not need to think of the “right order” in which to compute sub-solutions • Bottom-Up may be preferred… ➢ …when all sub-solutions will anyway need to be computed ➢ …because it is sometimes faster as it prevents recursive call overheads and unnecessary random memory accesses 373F19 - Nisarg Shah 10 Network Flow • Input ➢ A directed graph 퐺 = (푉, 퐸) ➢ Edge capacities 푐 ∶ 퐸 → ℝ≥0 ➢ Source node 푠, target node 푡 • Output ➢ Maximum “flow” from 푠 to 푡 373F19 - Nisarg Shah 11 Ford-Fulkerson Algorithm MaxFlow(퐺): // initialize: Set 푓 푒 = 0 for all 푒 in 퐺 // while there is an 푠-푡 path in 퐺푓: While 푃 = FindPath(s, t,Residual(퐺, 푓))!=None: 푓 = Augment(푓, 푃) UpdateResidual(퐺,푓) EndWhile Return 푓 373F19 - Nisarg Shah 12 Max Flow - Min Cut • Theorem: In any graph, the value of the maximum flow is equal to the capacity of the minimum cut. • Ford-Fulkerson can be used to find the min cut ∗ ➢ Find the max flow 푓 ∗ ➢ Let 퐴 = set of all nodes reachable from 푠 in residual graph 퐺푓∗ o Easy to compute using BFS ∗ ∗ ➢ Then (퐴 , 푉 ∖ 퐴 ) is min cut 373F19 - Nisarg Shah 13 LP, Standard Formulation 푛 푚 • Input: 푐, 푎1, 푎2, … , 푎푚 ∈ ℝ , 푏 ∈ ℝ ➢ There are 푛 variables and 푚 constraints • Goal: 373F19 - Nisarg Shah 14 LP, Standard Matrix Form 푛 푚 • Input: 푐, 푎1, 푎2, … , 푎푚 ∈ ℝ , 푏 ∈ ℝ ➢ There are 푛 variables and 푚 constraints • Goal: 373F19 - Nisarg Shah 15 Convert to Standard Form • What if the LP is not in standard form? ➢ Constraints that use ≥ o 푎푇푥 ≥ 푏 ⇔ −푎푇푥 ≤ −푏 ➢ Constraints that use equality o 푎푇푥 = 푏 ⇔ 푎푇푥 ≤ 푏, 푎푇푥 ≥ 푏 ➢ Objective function is a minimization o Minimize 푐푇푥 ⇔ Maximize −푐푇푥 ➢ Variable is unconstrained o 푥 with no constraint ⇔ Replace 푥 by two variables 푥′and 푥′′, replace every occurrence of 푥 with 푥′ − 푥′′, and add constraints 푥′ ≥ 0, 푥′′ ≥ 0 373F19 - Nisarg Shah 16 Duality Primal LP Dual LP • Weak duality theorem: 푇 푇 ➢ For any primal feasible 푥 and dual feasible 푦, 푐 푥 ≤ 푦 푏 • Strong duality theorem: ∗ ∗ 푇 ∗ ∗ 푇 ➢ For any primal optimal 푥 and dual optimal 푦 , 푐 푥 = 푦 푏 373F19 - Nisarg Shah 17 P • P (polynomial time) ➢ The class of all decision problems computable by a TM in polynomial time 373F19 - Nisarg Shah 18 NP • NP (nondeterministic polynomial time) ➢ The class of all decision problems for which a YES answer can be verified by a TM in polynomial time given polynomial length “advice” or “witness”. ➢ There is a polynomial-time verifier TM 푉 and another polynomial 푝 such that o For all YES inputs 푥, there exists 푦 with 푦 = 푝 푥 on which 푉(푥, 푦) returns YES o For all NO inputs 푥, 푉(푥, 푦) returns NO for every 푦 ➢ Informally: “Whenever the answer is YES, there’s a short proof of it.” 373F19 - Nisarg Shah 19 co-NP • co-NP ➢ Same as NP, except whenever the answer is NO, we want there to be a short proof of it 373F19 - Nisarg Shah 20 Reductions • Problem 퐴 is p-reducible to problem 퐵 if an “oracle” (subrouting) for 퐵 can be used to efficiently solve 퐴 ➢ You can solve 퐴 by making polynomially many calls to the oracle and doing additional polynomial computation 373F19 - Nisarg Shah 21 NP-completeness • NP-completeness ➢ A problem 퐵 is NP-complete if it is in NP and every problem 퐴 in NP is p-reducible to 퐵 ➢ Hardest problems in NP ➢ If one of them can be solved efficiently, every problem in NP can be solved efficiently, implying P=NP • Observation: ➢ If 퐴 is in NP, and some NP-complete problem 퐵 is p- reducible to 퐴, then 퐴 is NP-complete too o “If I could solve A, then I could solve B, then I could solve any problem in NP” 373F19 - Nisarg Shah 22 Review of Reductions • If you want to show that problem B is NP-complete • Step 1: Show that B is in NP ➢ Some polynomial-size advice should be sufficient to verify a YES instance in polynomial time ➢ No advice should work for a NO instance ➢ Usually, the solution of the “search version” of the problem works o But sometimes, the advice can be non-trivial o For example, to check LP optimality, one possible advice is the values of both primal and dual variables, as we saw in the last lecture 373F19 - Nisarg Shah 23 Review of Reductions • If you want to show that problem B is NP-complete • Step 2: Find a known NP-complete problem A and reduce it to B (i.e. show A ≤푝 B) ➢ This means taking an arbitrary instance of A, and solving it in polynomial time using an oracle for B o Caution 1: Remember the direction. You are “reducing known NP- complete problem to your current problem”. o Caution 2: The size of the B-instance you construct should be polynomial in the size of the original A-instance ➢ This would show that if B can be solved in polynomial time, then A can be as well ➢ Some reductions are trivial, some are notoriously tricky… 373F19 - Nisarg Shah 24 Approximation Algorithms • We focus on optimization problems ➢ Decision problem: “Is there…where…≥ 푘?” o E.g. “Is there an assignment which satisfies at least 푘 clauses of a given formula 휑?” ➢ Optimization problem: “Find…which maximizes…” o E.g. “Find an assignment which satisfies the maximum possible number of clauses from a given formula 휑.” ➢ Recall that if the decision problem is hard, then the optimization problem is hard too 373F19 - Nisarg Shah 25 Approximation Algorithms • There is a function 푃푟표푓푖푡 we want to maximize or a function 퐶표푠푡 we want to minimize • Given input instance 퐼… ➢ Our algorithm returns a solution 퐴퐿퐺(퐼) ➢ An optimal solution maximizing 푃푟표푓푖푡 or minimizing 퐶표푠푡 is 푂푃푇(퐼) ➢ Then, the approximation ratio of 퐴퐿퐺 on instance 퐼 is 푃푟표푓푖푡 푂푃푇 퐼 퐶표푠푡 퐿퐺 퐼 or 푃푟표푓푖푡 퐿퐺 퐼 퐶표푠푡 푂푃푇 퐼 373F19 - Nisarg Shah 26 Approximation Algorithms • Approximation ratio of 퐴퐿퐺 on instance 퐼 is 푃푟표푓푖푡 푂푃푇 퐼 퐶표푠푡 퐿퐺 퐼 or 푃푟표푓푖푡 퐿퐺 퐼 퐶표푠푡 푂푃푇 퐼 ➢ Note: These are defined to be ≥ 1 in each case. o 2-approximation = half the optimal profit / twice the optimal cost • 퐴퐿퐺 has worst-case 푐-approximation if for each instance 퐼… 1 푃푟표푓푖푡 퐴퐿퐺 퐼 ≥ ⋅ 푃푟표푓푖푡 푂푃푇 퐼 표푟 푐 퐶표푠푡 퐴퐿퐺 퐼 ≤ 푐 ⋅ 퐶표푠푡 푂푃푇 퐼 373F19 - Nisarg Shah 27 Local Search • A heuristic paradigm for solving complex problems ➢ Sometimes it might provably return an optimal solution ➢ But even if not, it might give a good approximation • Idea: ➢ Start with some solution 푆 ➢ While there is a “better” solution 푆′ in the local neighborhood of 푆 ➢ Switch to 푆’ • Need to define what is “better” and what is a “local neighborhood” 373F19 - Nisarg Shah 28 Randomized Algorithms Input Deterministic Algorithm Output Input Randomized Algorithm Output Randomness 373F19 - Nisarg Shah 29 Randomized Algorithms • Running time ➢ Sometimes, we want the algorithm to always take a small amount of time o Regardless of both the input and the random coin flips ➢ Sometimes, we want the algorithm to take a small amount of time in expectation o Expectation
Load more

Recommended publications
  • The Randomized Quicksort Algorithm
    Outline The Randomized Quicksort Algorithm K. Subramani1 1Lane Department of Computer Science and Electrical Engineering West Virginia University 7 February, 2012 Subramani Sample Analyses Outline Outline 1 The Randomized Quicksort Algorithm Subramani Sample Analyses The Randomized Quicksort Algorithm The Sorting Problem Problem Statement Given an array A of n distinct integers, in the indices A[1] through A[n], Subramani Sample Analyses The Randomized Quicksort Algorithm The Sorting Problem Problem Statement Given an array A of n distinct integers, in the indices A[1] through A[n], permute the elements of A, so that Subramani Sample Analyses The Randomized Quicksort Algorithm The Sorting Problem Problem Statement Given an array A of n distinct integers, in the indices A[1] through A[n], permute the elements of A, so that A[1] < A[2]... A[n]. Subramani Sample Analyses The Randomized Quicksort Algorithm The Sorting Problem Problem Statement Given an array A of n distinct integers, in the indices A[1] through A[n], permute the elements of A, so that A[1] < A[2]... A[n]. Note The assumption of distinctness simplifies the analysis. Subramani Sample Analyses The Randomized Quicksort Algorithm The Sorting Problem Problem Statement Given an array A of n distinct integers, in the indices A[1] through A[n], permute the elements of A, so that A[1] < A[2]... A[n]. Note The assumption of distinctness simplifies the analysis. It has no bearing on the running time. Subramani Sample Analyses The Randomized Quicksort Algorithm The Partition subroutine Function PARTITION(A,p,q) 1: {We partition the sub-array A[p,p + 1,...,q] about A[p].} 2: for (i = (p + 1) to q) do 3: if (A[i] < A[p]) then 4: Insert A[i] into bucket L.
    [Show full text]
  • Randomized Algorithms, Quicksort and Randomized Selection Carola Wenk Slides Courtesy of Charles Leiserson with Additions by Carola Wenk
    CMPS 2200 – Fall 2014 Randomized Algorithms, Quicksort and Randomized Selection Carola Wenk Slides courtesy of Charles Leiserson with additions by Carola Wenk CMPS 2200 Intro. to Algorithms 1 Deterministic Algorithms Runtime for deterministic algorithms with input size n: • Best-case runtime Attained by one input of size n • Worst-case runtime Attained by one input of size n • Average runtime Averaged over all possible inputs of size n CMPS 2200 Intro. to Algorithms 2 Deterministic Algorithms: Insertion Sort for j=2 to n { key = A[j] // insert A[j] into sorted sequence A[1..j-1] i=j-1 while(i>0 && A[i]>key){ A[i+1]=A[i] i-- } A[i+1]=key } • Best case runtime? • Worst case runtime? CMPS 2200 Intro. to Algorithms 3 Deterministic Algorithms: Insertion Sort Best-case runtime: O(n), input [1,2,3,…,n] Attained by one input of size n • Worst-case runtime: O(n2), input [n, n-1, …,2,1] Attained by one input of size n • Average runtime : O(n2) Averaged over all possible inputs of size n •What kind of inputs are there? • How many inputs are there? CMPS 2200 Intro. to Algorithms 4 Average Runtime • What kind of inputs are there? • Do [1,2,…,n] and [5,6,…,n+5] cause different behavior of Insertion Sort? • No. Therefore it suffices to only consider all permutations of [1,2,…,n] . • How many inputs are there? • There are n! different permutations of [1,2,…,n] CMPS 2200 Intro. to Algorithms 5 Average Runtime Insertion Sort: n=4 • Inputs: 4!=24 [1,2,3,4]0346 [4,1,2,3] [4,1,3,2] [4,3,2,1] [2,1,3,4]1235 [1,4,2,3] [1,4,3,2] [3,4,2,1] [1,3,2,4]1124 [1,2,4,3] [1,3,4,2] [3,2,4,1] [3,1,2,4]2455 [4,2,1,3] [4,3,1,2] [4,2,3,1] [3,2,1,4]3244 [2,1,4,3] [3,4,1,2] [2,4,3,1] [2,3,1,4]2333 [2,4,1,3] [3,1,4,2] [2,3,4,1] • Runtime is proportional to: 3 + #times in while loop • Best: 3+0, Worst: 3+6=9, Average: 3+72/24 = 6 CMPS 2200 Intro.
    [Show full text]
  • IBM Research Report Derandomizing Arthur-Merlin Games And
    H-0292 (H1010-004) October 5, 2010 Computer Science IBM Research Report Derandomizing Arthur-Merlin Games and Approximate Counting Implies Exponential-Size Lower Bounds Dan Gutfreund, Akinori Kawachi IBM Research Division Haifa Research Laboratory Mt. Carmel 31905 Haifa, Israel Research Division Almaden - Austin - Beijing - Cambridge - Haifa - India - T. J. Watson - Tokyo - Zurich LIMITED DISTRIBUTION NOTICE: This report has been submitted for publication outside of IBM and will probably be copyrighted if accepted for publication. It has been issued as a Research Report for early dissemination of its contents. In view of the transfer of copyright to the outside publisher, its distribution outside of IBM prior to publication should be limited to peer communications and specific requests. After outside publication, requests should be filled only by reprints or legally obtained copies of the article (e.g. , payment of royalties). Copies may be requested from IBM T. J. Watson Research Center , P. O. Box 218, Yorktown Heights, NY 10598 USA (email: [email protected]). Some reports are available on the internet at http://domino.watson.ibm.com/library/CyberDig.nsf/home . Derandomization Implies Exponential-Size Lower Bounds 1 DERANDOMIZING ARTHUR-MERLIN GAMES AND APPROXIMATE COUNTING IMPLIES EXPONENTIAL-SIZE LOWER BOUNDS Dan Gutfreund and Akinori Kawachi Abstract. We show that if Arthur-Merlin protocols can be deran- domized, then there is a Boolean function computable in deterministic exponential-time with access to an NP oracle, that cannot be computed by Boolean circuits of exponential size. More formally, if prAM ⊆ PNP then there is a Boolean function in ENP that requires circuits of size 2Ω(n).
    [Show full text]
  • January 30 4.1 a Deterministic Algorithm for Primality Testing
    CS271 Randomness & Computation Spring 2020 Lecture 4: January 30 Instructor: Alistair Sinclair Disclaimer: These notes have not been subjected to the usual scrutiny accorded to formal publications. They may be distributed outside this class only with the permission of the Instructor. 4.1 A deterministic algorithm for primality testing We conclude our discussion of primality testing by sketching the route to the first polynomial time determin- istic primality testing algorithm announced in 2002. This is based on another randomized algorithm due to Agrawal and Biswas [AB99], which was subsequently derandomized by Agrawal, Kayal and Saxena [AKS02]. We won’t discuss the derandomization in detail as that is not the main focus of the class. The Agrawal-Biswas algorithm exploits a different number theoretic fact, which is a generalization of Fermat’s Theorem, to find a witness for a composite number. Namely: Fact 4.1 For every a > 1 such that gcd(a, n) = 1, n is a prime iff (x − a)n = xn − a mod n. n Exercise: Prove this fact. [Hint: Use the fact that k = 0 mod n for all 0 < k < n iff n is prime.] The obvious approach for designing an algorithm around this fact is to use the Schwartz-Zippel test to see if (x − a)n − (xn − a) is the zero polynomial. However, this fails for two reasons. The first is that if n is not prime then Zn is not a field (which we assumed in our analysis of Schwartz-Zippel); the second is that the degree of the polynomial is n, which is the same as the cardinality of Zn and thus too large (recall that Schwartz-Zippel requires that values be chosen from a set of size strictly larger than the degree).
    [Show full text]
  • CMSC 420: Lecture 7 Randomized Search Structures: Treaps and Skip Lists
    CMSC 420 Dave Mount CMSC 420: Lecture 7 Randomized Search Structures: Treaps and Skip Lists Randomized Data Structures: A common design techlque in the field of algorithm design in- volves the notion of using randomization. A randomized algorithm employs a pseudo-random number generator to inform some of its decisions. Randomization has proved to be a re- markably useful technique, and randomized algorithms are often the fastest and simplest algorithms for a given application. This may seem perplexing at first. Shouldn't an intelligent, clever algorithm designer be able to make better decisions than a simple random number generator? The issue is that a deterministic decision-making process may be susceptible to systematic biases, which in turn can result in unbalanced data structures. Randomness creates a layer of \independence," which can alleviate these systematic biases. In this lecture, we will consider two famous randomized data structures, which were invented at nearly the same time. The first is a randomized version of a binary tree, called a treap. This data structure's name is a portmanteau (combination) of \tree" and \heap." It was developed by Raimund Seidel and Cecilia Aragon in 1989. (Remarkably, this 1-dimensional data structure is closely related to two 2-dimensional data structures, the Cartesian tree by Jean Vuillemin and the priority search tree of Edward McCreight, both discovered in 1980.) The other data structure is the skip list, which is a randomized version of a linked list where links can point to entries that are separated by a significant distance. This was invented by Bill Pugh (a professor at UMD!).
    [Show full text]
  • Randomized Algorithms
    Randomized Algorithms Prabhakar Raghavan IBM Almaden Research Center San Jose CA Typ eset byFoilT X E Deterministic Algorithms INPUT ALGORITHM OUTPUT Goal To prove that the algorithm solves the problem correctly always and quickly typically the number of steps should be p olynomial in the size of the input Typ eset byFoilT X E Randomized Algorithms INPUT ALGORITHM OUTPUT RANDOM NUMBERS In addition to input algorithm takes a source of random numbers and makes random choices during execution Behavior can vary even on a xed input Typ eset byFoilT X E Randomized Algorithms INPUT ALGORITHM OUTPUT RANDOM NUMBERS Design algorithm analysis to show that this b ehavior is likely to be good on every input The likeliho o d is over the random numbers only Typ eset byFoilT X E Not to be confused with the Probabilistic Analysis of Algorithms RANDOM INPUT ALGORITHM OUTPUT DISTRIBUTION Here the input is assumed to be from a probability distribution Show that the algorithm works for most inputs Typ eset byFoilT X E Monte Carlo and Las Vegas A Monte Carlo algorithm runs for a xed number of steps and pro duces an answer that is correct with probability A Las Vegas algorithm always pro duces the correct answer its running time is a random variable whose exp ectation is bounded say by a polynomial Typ eset byFoilT X E Monte Carlo and Las Vegas These probabilitiesexp ectations are only over the random choices made by the algorithm indep endent of the input Thus indep endent rep etitions of Monte Carlo algorithms drive down the failure probability
    [Show full text]
  • Randomized Algorithms How Do We Evaluate This?
    Analyzing Algorithms Goal: “Runs fast on typical real problem instances” Randomized Algorithms How do we evaluate this? Example: Binary search Given a sorted array, determine if the array contains the number 157? 2 3 Complexity " Measuring efficiency T! analysis n! Time ≈ # of instructions executed in a simple Problem size n programming language Best-case complexity: min # steps algorithm only simple operations (+,*,-,=,if,call,…) takes on any input of size n each operation takes one time step Average-case complexity: avg # steps algorithm each memory access takes one time step takes on inputs of size n Worst-case complexity: max # steps algorithm takes on any input of size n 4 5 1 Complexity The complexity of an algorithm associates a number Complexity T(n), the worst-case time the algorithm takes on problems of size n, with each problem size n. T(n)! Mathematically, T: N+ → R+ ! I.e., T is a function that maps positive integers (problem Time sizes) to positive real numbers (number of steps). Problem size ! 6 7 Simple Example Complexity Array of bits. The complexity of an algorithm associates a number I promise you that either they are all 1’s or # 0’s T(n), the worst-case time the algorithm takes on and # 1’s. problems of size n, with each problem size n. Give me a program that will tell me which it is. For randomized algorithms, look at worst-case value of E(T), where the Best case? expectation is taken over randomness in Worst case? algorithm. Neat idea: use randomization to reduce the worst case 8 9 2 Quicksort Analysis of Quicksort Worst case number of comparisons: n Sorting algorithm (assume for now all elements 2 distinct) ✓ ◆ How can we use randomization to improve running Given array of some length n time? If n = 0 or 1, halt Pick random element as a pivot each step Else pick element p of array as “pivot” Split array into subarrays <p, > p => Randomized algorithm Recursively sort elements < p Recursively sort elements > p 10 11 Analysis of Randomized Quicksort Analysis of Randomized Quicksort" Quicksort with random pivots Fix pair i,j.
    [Show full text]
  • CS265/CME309: Randomized Algorithms and Probabilistic Analysis Lecture #1:Computational Models, and the Schwartz-Zippel Randomized Polynomial Identity Test
    CS265/CME309: Randomized Algorithms and Probabilistic Analysis Lecture #1:Computational Models, and the Schwartz-Zippel Randomized Polynomial Identity Test Gregory Valiant,∗ updated by Mary Wootters September 7, 2020 1 Introduction Welcome to CS265/CME309!! This course will revolve around two intertwined themes: • Analyzing random structures, including a variety of models of natural randomized processes, including random walks, and random graphs/networks. • Designing random structures and algorithms that solve problems we care about. By the end of this course, you should be able to think precisely about randomness, and also appreciate that it can be a powerful tool. As we will see, there are a number of basic problems for which extremely simple randomized algorithms outperform (both in theory and in practice) the best deterministic algorithms that we currently know. 2 Computational Model During this course, we will discuss algorithms at a high level of abstraction. Nonetheless, it's helpful to begin with a (somewhat) formal model of randomized computation just to make sure we're all on the same page. Definition 2.1 A randomized algorithm is an algorithm that can be computed by a Turing machine (or random access machine), which has access to an infinite string of uniformly random bits. Equivalently, it is an algorithm that can be performed by a Turing machine that has a special instruction “flip-a-coin”, which returns the outcome of an independent flip of a fair coin. ∗©2019, Gregory Valiant. Not to be sold, published, or distributed without the authors' consent. 1 We will never describe algorithms at the level of Turing machine instructions, though if you are ever uncertain whether or not an algorithm you have in mind is \allowed", you can return to this definition.
    [Show full text]
  • Lecture 3: Randomness in Computation
    Great Ideas in Theoretical Computer Science Summer 2013 Lecture 3: Randomness in Computation Lecturer: Kurt Mehlhorn & He Sun Randomness is one of basic resources and appears everywhere. In computer science, we study randomness due to the following reasons: 1. Randomness is essential for computers to simulate various physical, chemical and biological phenomenon that are typically random. 2. Randomness is necessary for Cryptography, and several algorithm design tech- niques, e.g. sampling and property testing. 3. Even though efficient deterministic algorithms are unknown for many problems, simple, elegant and fast randomized algorithms are known. We want to know if these randomized algorithms can be derandomized without losing the efficiency. We address these in this lecture, and answer the following questions: (1) What is randomness? (2) How can we generate randomness? (3) What is the relation between randomness and hardness and other disciplines of computer science and mathematics? 1 What is Randomness? What are random binary strings? One may simply answer that every 0/1-bit appears with the same probability (50%), and hence 00000000010000000001000010000 is not random as the number of 0s is much more than the number of 1s. How about this sequence? 00000000001111111111 The sequence above contains the same number of 0s and 1s. However most people think that it is not random, as the occurrences of 0s and 1s are in a very unbalanced way. So we roughly call a string S 2 f0; 1gn random if for every pattern x 2 f0; 1gk, the numbers of the occurrences of every x of the same length are almost the same.
    [Show full text]
  • Randomized Algorithms: Quicksort and Selection
    Randomized Algorithms: Quicksort and Selection Version of September 6, 2016 Randomized Algorithms: Quicksort and Selection Version1 / 30 of September 6, 2016 Outline Outline: Quicksort Average-Case Analysis of QuickSort Randomized quicksort Selection The selection problem First solution: Selection by sorting Randomized Selection Randomized Algorithms: Quicksort and Selection Version2 / 30 of September 6, 2016 Quicksort: Review Quicksort(A; p; r) begin if p < r then q = Partition(A; p; r); Quicksort(A; p; q − 1); Quicksort(A; q + 1; r); end end Partition(A; p; r) reorders items in A[p ::: r]; items < A[r] are to its left; items > A[r] to its right. Showed that if input is a random input (permutation) of n items, then average running time is O(n log n) Randomized Algorithms: Quicksort and Selection Version3 / 30 of September 6, 2016 Average Case Analysis of Quicksort Formally, the average running time can be defined as follows: In is the set of all n! inputs of size n I 2 In is any particular size-n input R(I ) is the running time of the algorithm on input I Then, the average running time over the random inputs is X 1 X Pr(I )R(I ) = R(I ) = O(n log n) n! I 2In I 2In Only fact that was used was that A[r] was a random item in A[p ::: r], i.e., the partition item is equally likely to be any item in the subset. Randomized Algorithms: Quicksort and Selection Version4 / 30 of September 6, 2016 Outline Outline: Quicksort Average-Case Analysis of QuickSort Randomized Quicksort Selection The selection problem First solution: Selection by sorting Randomized Selection Randomized Algorithms: Quicksort and Selection Version5 / 30 of September 6, 2016 Randomized-Partition(A; p; r) Idea: In the algorithm Partition(A; p; r), A[r] is always used as the pivot x to partition the array A[p::r] In the algorithm Randomized-Partition(A; p; r), we randomly choose j, p ≤ j ≤ r, and use A[j] as pivot Idea is that if we choose randomly, then the chance that we get unlucky every time is extremely low.
    [Show full text]
  • Lecture 01: Randomized Algorithm for Reachability Problem
    Lecture 01: Randomized Algorithm for Reachability Problem Topics in Pseudorandomness and Complexity (Spring 2018) Rutgers University Swastik Kopparty Scribes: Neelesh Kumar and Justin Semonsen The lecture is an introduction to randomized algorithms, and builds a motivation for using such algorithms by taking the example of reachability problem. 1 Deterministic Algorithm vs. Randomized Algorithm We are given some function f : f0; 1gn ! f0; 1g, and we wish to compute f(x) quickly. A de- terministic algorithm A for computing f is one which always produces the same A(x) = f(x), 8x 2 f0; 1gn. On the other hand, a randomized algorithm A for computing f takes two arguments- the input x and a random string r such that 8x 2 f0; 1gn, P rr2f0;1gn [A(x; r) = f(x)] ≥ 0:9 (1) i.e. with high probability over choices of r, A produces the correct result. It should be noted that the algorithm A itself may not have inherent randomness, rather it might make random calls deterministically. It should also be noted that (1) should hold true for all inputs x, and for most choices of r. Some examples of such algorithms are polynomial identity testing, primality testing (prior to 2002, the only way to do this was using a randomized algorithm), random sampling, etc. 2 Motivating Example Let us consider an interesting example to motivate the use of randomized algorithm. We are given as input a graph G and two of its vertices s and t. We wish to know if there exists a path from s to t in G.
    [Show full text]
  • Linear Advice for Randomized Logarithmic Space
    Linear Advice for Randomized Logarithmic Space Lance Fortnow1 and Adam R. Klivans2? 1 Department of Computer Science University of Chicago Chicago, IL 60637 [email protected] 2 Department of Computer Science The University of Texas at Austin Austin, TX 78712 [email protected] Abstract. We show that RL ⊆ L/O(n), i.e., any language computable in randomized logarithmic space can be computed in deterministic log- arithmic space with a linear amount of non-uniform advice. To prove our result we use an ultra-low space walk on the Gabber-Galil expander graph due to Gutfreund and Viola. 1 Introduction The question of whether RL, randomized logarithmic space, can be simulated in L, deterministic logarithmic space, remains a central challenge in complexity- theory. The best known deterministic simulation of randomized logarithmic space is due to Saks and Zhou [16] who, building on seminal work due to Nisan [13], proved that BPL ⊆ L3/2. Recently in a breakthrough result, Rein- gold [14] proved that the s-t connectivity problem on undirected graphs could be solved in L; this implies SL, the symmetric analogue of NL, equals L. The possibility of extending his techniques to prove RL = L has subsequently been investigated by Reingold, Trevisan and Vadhan [15]. 1.1 Randomness and Non-Uniformity The relationship between randomness and non-uniformity is a topic of funda- mental importance in complexity theory. Derandomizing complexity classes fre- quently involves a consideration of the smallest non-uniform complexity class or circuit family which contains a particular randomized class. Adleman’s well known result on BPP [1], for example, shows that BPP can be computed by polynomial-size circuits.
    [Show full text]