DIVIDE AND CONQUER II ‣ master theorem ‣ integer multiplication ‣ matrix multiplication ‣ convolution and FFT Lecture slides by Kevin Wayne Copyright © 2005 Pearson-Addison Wesley http://www.cs.princeton.edu/~wayne/kleinberg-tardos Last updated on 2/6/21 8:29 PM DIVIDE AND CONQUER II ‣ master theorem ‣ integer multiplication ‣ matrix multiplication ‣ convolution and FFT SECTIONS 4.4–4.6 Divide-and-conquer recurrences Goal. Recipe for solving common divide-and-conquer recurrences: n T (n)=aT + f(n) b with T(0) = 0 and T(1) = Θ(1). Terms. ・a ≥ 1 is the number of subproblems. ・b ≥ 2 is the factor by which the subproblem size decreases. ・f (n) ≥ 0 is the work to divide and combine subproblems. T (n) Recursion tree. [ assuming n is a power of b ] ・a = branching factor. ・ai = number of subproblems at level i. 1 + log n levels. ・ b T (n / b) T (n / b) ... T (n / b) ・n / bi = size of subproblem at level i. ... ... ... 3 Divide-and-conquer recurrences: recursion tree c Suppose T (n) satisfies T (n) = a T (n / b) + n with T (1) = 1, for n a power of b. T (n) nc c T (n / b) T (n / b) ... T (n / b) a (n / b) 1 + logb n 2 2 ... 2 2 2 ... 2 2 2 ... 2 2 2 c T (n / b ) T (n / b ) T (n / b ) T (n / b ) T (n / b ) T (n / b ) T (n / b ) T (n / b ) T (n / b ) a (n / b ) i i c ⋮ a (n / b ) ⋮ logb a T (1) T (1) T (1) T (1) T (1) T (1) T (1) T (1) T (1) T (1) ... T (1) T (1) T (1) n alogbn = nlogba logb n r = a / b c T (n)=nc ri 4 <latexit sha1_base64="dMY5/xq94vZUiORXKa1OXPmRnaE=">AAACU3icbVDLSgMxFM2Mr/quunQTLIK6KDMiqIgguHFZwarQaYdMeluDmWRI7ohlmH/wa9zqVwj+iwvT2oWtHkg4nHNvbu5JMiksBsGn58/Mzs0vVBaXlldW19arG5u3VueGQ5Nrqc19wixIoaCJAiXcZwZYmki4Sx4vh/7dExgrtLrBQQbtlPWV6AnO0Elx9eBmT+3T6IyeDy/V4TSyeRoX4jwoO0UkdT9OqCqp6Yi4WgvqwQj0LwnHpEbGaMQb3mbU1TxPQSGXzNpWGGTYLphBwSWUS1FuIWP8kfWh5ahiKdh2MVqqpLtO6dKeNu4opCP1d0fBUmsHaeIqU4YPdtobiv95rRx7J+1CqCxHUPxnUC+XFDUdJkS7wgBHOXCEcSPcXyl/YIZxdDlOTBm9nQGf2KR4zpXgugtTqsRnNKx0KYbTmf0lzcP6aT28PqpdnIzjrJBtskP2SEiOyQW5Ig3SJJy8kFfyRt69D+/L9/3Zn1LfG/dskQn4q99GQrOQ</latexit> i=0 Divide-and-conquer recurrences: recursion tree analysis c Suppose T (n) satisfies T (n) = a T (n / b) + n with T (1) = 1, for n a power of b. c Let r = a / b . Note that r < 1 iff c > logb a. c cost dominated Θ(n ) r<1 c > logb a by cost of root logb n c i c cost evenly T (n)=n r = Θ(n log n) r =1 c = logb a distributed in tree i=0 log a cost dominated Θ(n b ) r>1 c < logb a <latexit sha1_base64="NEShmpjKNsOBbDerTDYmROKsym4=">AAAC/nicbVFNj9MwEHXCxy7lq7scuYyoQN1LlSAkdlWKVuLCcZFadqU6jRx32nrrOJHtoFZRDvwaTogrf4QD/wanm5VotyPZenrz3sx4nORSGBsEfz3/3v0HDw8OH7UeP3n67Hn76PiryQrNccQzmemrhBmUQuHICivxKtfI0kTiZbL8VOcvv6E2IlNDu84xStlciZngzDoqbv8ZdtUJ0D4M6ktNOFBTpHEpBkE1KanM5nECqgI9EbeqFk1wLlTJXVtTtWifDhdoWdeZT+ANUIsrW4pZVes1fIAQKB2HmEZbUqhrg9rjGOx33A7Dqj2ejxC2KKppM1Tc7gS9YBNwF4QN6JAmLuIj75hOM16kqCyXzJhxGOQ2Kpm2gkt0rywM5owv2RzHDiqWoonKzf4reO2YKcwy7Y6ysGH/d5QsNWadJk6ZMrswu7ma3JcbF3Z2GpVC5YVFxW8azQoJNoP6M2EqNHIr1w4wroWbFfiCacat+/KtLpvaOfKtl5SrQgmeTXGHlXZlNau3GO7u7C4Yve2d9cIv7zrnp806D8lL8op0SUjek3PymVyQEeFe32Petbf0v/s//J/+rxup7zWeF2Qr/N//AAuD6zQ=</latexit> by cost of leaves Geometric series. ・If 0 < r < 1, then 1 + r + r2 + r3 + … + rk ≤ 1 / (1 − r). ・If r = 1, then 1 + r + r2 + r3 + … + rk = k + 1. ・If r > 1, then 1 + r + r2 + r3 + … + rk = (rk+1 − 1) / (r − 1). 5 Divide-and-conquer recurrences: master theorem Master theorem. Let a ≥ 1, b ≥ 2, and c ≥ 0 and suppose that T (n) is a function on the non-negative integers that satisfies the recurrence n T (n)=aT + Θ(nc) <latexit sha1_base64="RRWvKmBD1IL0biczVisiLGro+Bk=">AAACb3icbVBtSxtBEN6c2lq1NeoXQZClQUiohLtSqEUEQRA/KiRVyMUwt5lLFvf2jt25YjjuH/lr/Cb6K/wF3bx8MNGB3X145pmdmSfKlLTk+08Vb2l55dPn1S9r6xtfv21Wt7b/2jQ3AtsiVam5icCikhrbJEnhTWYQkkjhdXR3Ns5f/0NjZapbNMqwm8BAy1gKIEf1quetum7w8JifjC/g4SFvhQpj4vUwNiAKXRZRyUMjB0PiTnnoFD+mT9gaIkFd34pGr1rzm/4k+HsQzECNzeKyt1XZDvupyBPUJBRY2wn8jLoFGJJCYbkW5hYzEHcwwI6DGhK03WKycMkPHNPncWrc0cQn7NuKAhJrR4kb/CABGtrF3Jj8KNfJKT7qFlJnOaEW00ZxrjilfOwe70uDgtTIARBGulm5GILziZzHc10mf2co5jYp7nMtRdrHBVbRPRkonYvBomfvQftn808zuPpVOz2a2bnK9th3VmcB+81O2QW7ZG0m2AN7ZM/spfLq7Xr7Hp9KvcqsZofNhdf4D8cluuQ=</latexit> b with T(0) = 0 and T(1) = Θ(1), where n / b means either ⎣n / b⎦ or ⎡n / b⎤. Then, c Case 1. If c > logb a, then T (n) = Θ(n ). c Case 2. If c = logb a, then T (n) = Θ(n log n). log a Case 3. If c < logb a, then T (n) = Θ(n b ). Pf sketch. ・Prove when b is an integer and n is an exact power of b. ・Extend domain of recurrences to reals (or rationals). ・Deal with floors and ceilings. at most 2 extra levels in recursion tree n/b /b /b < n/b3 +(1/b2 +1/b + 1) 3 n/b +2 6 <latexit sha1_base64="32puyXOtqPUNNUvXXxzCTvaPp5o=">AAACpHicbZHfT9swEMedbEDX8aOFx71YqyggpDbpJq0TPCDxwgsS08iK1ITKca+theNE9mWiivqH7nH/CU4oEi2cFN9H37uz46/jTAqDnvfPcT983Njcqn2qf97e2d1rNPf/mDTXHAKeylTfxcyAFAoCFCjhLtPAkljCIH64LOuDv6CNSNUtzjOIEjZVYiI4QyuNGkUoOQhJV5PqxjTUFXbpe/iSzmj7vFztwP03ekqPfQs9CzaX6wkNw2HHhySqt+3m8Lq3N2q0vI5XBX0L/hJaZBk3o6azH45TniegkEtmzND3MowKplFwCYt6mBvIGH9gUxhaVCwBExWVSwt6aJUxnaTafgpppb6eKFhizDyJbWfCcGbWa6X4Xm2Y46QfFUJlOYLizwdNckkxpaXldCw0cJRzC4xrYf+V8hnTjKN9mJVTqr0z4Cs3KR5zJXg6hjVV4iNqtrAu+uuevYWg1/nZ8X99b130l3bWyBfylRwTn/wgF+SK3JCAcPLf2XIaTtM9cq/d327w3Oo6y5kDshLu/RM03scN</latexit> ≤ Divide-and-conquer recurrences: master theorem Master theorem. Let a ≥ 1, b ≥ 2, and c ≥ 0 and suppose that T (n) is a function on the non-negative integers that satisfies the recurrence n T (n)=aT + Θ(nc) <latexit sha1_base64="RRWvKmBD1IL0biczVisiLGro+Bk=">AAACb3icbVBtSxtBEN6c2lq1NeoXQZClQUiohLtSqEUEQRA/KiRVyMUwt5lLFvf2jt25YjjuH/lr/Cb6K/wF3bx8MNGB3X145pmdmSfKlLTk+08Vb2l55dPn1S9r6xtfv21Wt7b/2jQ3AtsiVam5icCikhrbJEnhTWYQkkjhdXR3Ns5f/0NjZapbNMqwm8BAy1gKIEf1quetum7w8JifjC/g4SFvhQpj4vUwNiAKXRZRyUMjB0PiTnnoFD+mT9gaIkFd34pGr1rzm/4k+HsQzECNzeKyt1XZDvupyBPUJBRY2wn8jLoFGJJCYbkW5hYzEHcwwI6DGhK03WKycMkPHNPncWrc0cQn7NuKAhJrR4kb/CABGtrF3Jj8KNfJKT7qFlJnOaEW00ZxrjilfOwe70uDgtTIARBGulm5GILziZzHc10mf2co5jYp7nMtRdrHBVbRPRkonYvBomfvQftn808zuPpVOz2a2bnK9th3VmcB+81O2QW7ZG0m2AN7ZM/spfLq7Xr7Hp9KvcqsZofNhdf4D8cluuQ=</latexit> b with T(0) = 0 and T(1) = Θ(1), where n / b means either ⎣n / b⎦ or ⎡n / b⎤. Then, c Case 1. If c > logb a, then T (n) = Θ(n ). c Case 2. If c = logb a, then T (n) = Θ(n log n). log a Case 3. If c < logb a, then T (n) = Θ(n b ). Extensions. ・Can replace Θ with O everywhere. ・Can replace Θ with Ω everywhere. ・Can replace initial conditions with T(n) = Θ(1) for all n ≤ n0 and require recurrence to hold only for all n > n0. 7 Divide-and-conquer recurrences: master theorem Master theorem. Let a ≥ 1, b ≥ 2, and c ≥ 0 and suppose that T (n) is a function on the non-negative integers that satisfies the recurrence n T (n)=aT + Θ(nc) <latexit sha1_base64="RRWvKmBD1IL0biczVisiLGro+Bk=">AAACb3icbVBtSxtBEN6c2lq1NeoXQZClQUiohLtSqEUEQRA/KiRVyMUwt5lLFvf2jt25YjjuH/lr/Cb6K/wF3bx8MNGB3X145pmdmSfKlLTk+08Vb2l55dPn1S9r6xtfv21Wt7b/2jQ3AtsiVam5icCikhrbJEnhTWYQkkjhdXR3Ns5f/0NjZapbNMqwm8BAy1gKIEf1quetum7w8JifjC/g4SFvhQpj4vUwNiAKXRZRyUMjB0PiTnnoFD+mT9gaIkFd34pGr1rzm/4k+HsQzECNzeKyt1XZDvupyBPUJBRY2wn8jLoFGJJCYbkW5hYzEHcwwI6DGhK03WKycMkPHNPncWrc0cQn7NuKAhJrR4kb/CABGtrF3Jj8KNfJKT7qFlJnOaEW00ZxrjilfOwe70uDgtTIARBGulm5GILziZzHc10mf2co5jYp7nMtRdrHBVbRPRkonYvBomfvQftn808zuPpVOz2a2bnK9th3VmcB+81O2QW7ZG0m2AN7ZM/spfLq7Xr7Hp9KvcqsZofNhdf4D8cluuQ=</latexit> b with T(0) = 0 and T(1) = Θ(1), where n / b means either ⎣n / b⎦ or ⎡n / b⎤. Then, c Case 1. If c > logb a, then T (n) = Θ(n ). c Case 2. If c = logb a, then T (n) = Θ(n log n). log a Case 3. If c < logb a, then T (n) = Θ(n b ). Ex. [Case 1] T (n) = 3 T(⎣n / 2⎦) + 5 n. ・a = 3, b = 2, c = 1 < logb a = 1.5849.... ・T(n) = Θ(nlog23) = O(n1.58). 8 Divide-and-conquer recurrences: master theorem Master theorem. Let a ≥ 1, b ≥ 2, and c ≥ 0 and suppose that T (n) is a function on the non-negative integers that satisfies the recurrence n T (n)=aT + Θ(nc) <latexit sha1_base64="RRWvKmBD1IL0biczVisiLGro+Bk=">AAACb3icbVBtSxtBEN6c2lq1NeoXQZClQUiohLtSqEUEQRA/KiRVyMUwt5lLFvf2jt25YjjuH/lr/Cb6K/wF3bx8MNGB3X145pmdmSfKlLTk+08Vb2l55dPn1S9r6xtfv21Wt7b/2jQ3AtsiVam5icCikhrbJEnhTWYQkkjhdXR3Ns5f/0NjZapbNMqwm8BAy1gKIEf1quetum7w8JifjC/g4SFvhQpj4vUwNiAKXRZRyUMjB0PiTnnoFD+mT9gaIkFd34pGr1rzm/4k+HsQzECNzeKyt1XZDvupyBPUJBRY2wn8jLoFGJJCYbkW5hYzEHcwwI6DGhK03WKycMkPHNPncWrc0cQn7NuKAhJrR4kb/CABGtrF3Jj8KNfJKT7qFlJnOaEW00ZxrjilfOwe70uDgtTIARBGulm5GILziZzHc10mf2co5jYp7nMtRdrHBVbRPRkonYvBomfvQftn808zuPpVOz2a2bnK9th3VmcB+81O2QW7ZG0m2AN7ZM/spfLq7Xr7Hp9KvcqsZofNhdf4D8cluuQ=</latexit> b with T(0) = 0 and T(1) = Θ(1), where n / b means either ⎣n / b⎦ or ⎡n / b⎤. Then, c Case 1. If c > logb a, then T (n) = Θ(n ). c Case 2. If c = logb a, then T (n) = Θ(n log n). log a Case 3. If c < logb a, then T (n) = Θ(n b ). ok to intermix floor and ceiling Ex. [Case 2] T (n) = T(⎣n / 2⎦) + T(⎡n / 2⎤) + 17 n. ・a = 2, b = 2, c = 1 = logb a ・T (n) = Θ(n log n). 9 Divide-and-conquer recurrences: master theorem Master theorem. Let a ≥ 1, b ≥ 2, and c ≥ 0 and suppose that T (n) is a function on the non-negative integers that satisfies the recurrence n T (n)=aT + Θ(nc) <latexit sha1_base64="RRWvKmBD1IL0biczVisiLGro+Bk=">AAACb3icbVBtSxtBEN6c2lq1NeoXQZClQUiohLtSqEUEQRA/KiRVyMUwt5lLFvf2jt25YjjuH/lr/Cb6K/wF3bx8MNGB3X145pmdmSfKlLTk+08Vb2l55dPn1S9r6xtfv21Wt7b/2jQ3AtsiVam5icCikhrbJEnhTWYQkkjhdXR3Ns5f/0NjZapbNMqwm8BAy1gKIEf1quetum7w8JifjC/g4SFvhQpj4vUwNiAKXRZRyUMjB0PiTnnoFD+mT9gaIkFd34pGr1rzm/4k+HsQzECNzeKyt1XZDvupyBPUJBRY2wn8jLoFGJJCYbkW5hYzEHcwwI6DGhK03WKycMkPHNPncWrc0cQn7NuKAhJrR4kb/CABGtrF3Jj8KNfJKT7qFlJnOaEW00ZxrjilfOwe70uDgtTIARBGulm5GILziZzHc10mf2co5jYp7nMtRdrHBVbRPRkonYvBomfvQftn808zuPpVOz2a2bnK9th3VmcB+81O2QW7ZG0m2AN7ZM/spfLq7Xr7Hp9KvcqsZofNhdf4D8cluuQ=</latexit> b with