Computations (DAA, M.Tech + Ph.D.)

By:

Sunil Kumar Singh, PhD Assistant Professor, Department of Computer Science and Information Technology

School of Computational Sciences, Information and Communication Technology, Mahatma Gandhi Central University, Motihari

27-04-2020 Bihar, India-845401 1 Outline • Matrix Computation • Strassen’s Algorithm • Divide an conquer paradigm for multiplication • Inverse of a triangular matrix • Conclusion • References

27-04-2020 2 Matrix Computations

Input: 퐴 = 푎푖푗 ,퐵 = 푏푖푗 output: 퐶 = 푐푖푗 = 퐴. 퐵 푤ℎ푒푟푒 푖, 푗 = {1,2,3, … … 푛}

푐11 푐12 … 푐1푛 푎11 푎12 … 푎1푛 푏11 푏12 … 푏1푛 푐 푐 … 푐22 푎 푎 … 푎22 푏 푏 … 푏 21 22 = 21 22 . 21 22 22 ⋮ ⋮ ⋱ ⋮ ⋮ ⋮ ⋱ ⋮ ⋮ ⋮ ⋱ ⋮ 푐푛1 푐푛2 … 푐푛푛 푎푛1 푎푛2 … 푎푛푛 푏푛1 푏푛2 … 푏푛푛

푛

푐푖푗 = ෍ 푎푖푘. 푏푘푗 푘=1

3 28-04-2020 Standard Algorithm

푁 푁 푁 3 3 푇 푁 = σ푖=1 σ푗=1 σ푘=1 푐=c 푁 = 푂(푁 )

4 27-04-2020 Divide-and-Conquer • Divide-and conquer is a general algorithm design paradigm:

✓ Divide: divide the input data S in two or more disjoint subsets

푆1, 푆2 …

✓ Solve the subproblems recursively

✓ Conquer: combine the solutions for 푆1, 푆2 … into a solution for 푆

• The base case for the recursion are sub problems of constant size

• Analysis can be done using recurrence equations

5 28-04-2020 through divide and conquer approach 푛 푛 1. Divide: Partition A and B into 푋 submatrices. Form terms 2 2 to be multiplied using + and –

푛 푛 2. Conquer: Perform 7 multiplications of 푋 submatrices 2 2 recursively.

푛 푛 3. Combine: Form C using + and – on 푋 submatrices 2 2

6 28-04-2020 Matrix multiplication through divide and conquer approach 푛 × 푛 푚푎푡푟푖푥 = 2 × 2 matrix of (푛/2) × (푛/2) submatrices:

푟 ⋮ 푠 푎 ⋮ 푏 푒 ⋮ 푓 ⋯ ⋮ ⋯ = ⋯ ⋮ ⋯ . ⋯ ⋮ ⋯ 푡 ⋮ 푢 푐 ⋮ 푑 푔 ⋮ ℎ

퐶 = 퐴. 퐵

푟 = 푎푒 + 푏푔 푠 = 푎푓 + 푏ℎ n n 8 multiplications and 4 adds of X 푡 = 푐푒 + 푑푔 2 2 푢 = 푐푓 + 푑ℎ

7 28-04-2020 Matrix multiplication through divide and conquer approach 푛 푇 푛 = 8푇 + Θ(푛2) 2

#submatrices work adding submatrices submatrix size

Complexity: 푇 푛 = Θ(푛3)

8 28-04-2020 Strassen’s Algorithm • In , the Strassen algorithm, named after , is an algorithm for matrix multiplication.

• It is faster than standard matrix and is useful in practice for large matrices.

• But it would be slower than the fastest known algorithms for extremely large matrices.

9 28-04-2020 Strassen’s Idea • Multiply 2 × 2 matrices with only 7 recursive multiplications.

퐶 퐶 퐴 퐴 퐵 퐵 11 12 = 11 12 . 11 12 퐶21 퐶22 퐴21 퐴22 퐵21 퐵22

푷ퟏ = (푨ퟏퟏ + 푨ퟐퟐ)(푩ퟏퟏ + 푩ퟐퟐ) 푪ퟏퟏ = 푷ퟏ + 푷ퟒ − 푷ퟓ + 푷ퟕ

푷ퟐ = 푨ퟐퟏ + 푨ퟐퟐ ∗ (푩ퟏퟏ) 푪ퟏퟐ = 푷ퟑ + 푷ퟓ

푷ퟑ = 푨ퟏퟏ ∗ (푩ퟏퟐ − 푩ퟐퟐ) 푪ퟐퟏ = 푷ퟐ + 푷ퟒ

푷ퟒ = (푨ퟐퟐ)(푩ퟏퟏ + 푩ퟐퟐ) 푪ퟏퟏ = 푷ퟏ + 푷ퟑ − 푷ퟐ + 푷ퟔ

푷ퟓ = 푨ퟏퟏ + 푨ퟐퟐ ∗ (푩ퟐퟐ)

푷ퟔ = 푨ퟐퟏ − 푨ퟏퟏ ∗ (푩ퟏퟏ + 푩ퟏퟐ)

푷ퟕ = 푨ퟏퟐ − 푨ퟐퟐ ∗ (푩ퟐퟏ + 푩ퟐퟐ) 10 28-04-2020 Analysis

푏 푛 ≤ 2 푇 푛 = ቐ 푛 7푇 + Θ(푛2) 푛 > 2 2 • When we apply the masters theorem:

푇 푛 = 푂(푛log2 7)

푇 푛 = 푂(푛2.81) • The number 2.81 may not seem much smaller than 3, but because the difference is in the exponent, the impact on running time is significant. In fact, Strassen’s algorithm beats the ordinary algorithm on today’s machines for 푛 ≥ 32 or so. 11 28-04-2020 Inverse of a Matrix • Inverse of an 푛 × 푛 matrix A to be the 푛 × 푛 matrix, denoted by 퐴−1(if it exists), such that −1 −1 퐴퐴 = 퐼푛 = 퐴 퐴 • for example 1 1 −1 0 1 = 1 0 1 −1 • Many nonzero 푛 × 푛 matrices do not have inverse. A matrix without an inverse is called noninvertible, or singular. • An example of a non-zero singular matrix is 1 0 1 0 • If a matrix has an inverse, it is called invertible, or nonsingular.

12 28-04-2020 Inverse of a Matrix • If A and B are nonsingular 푛 × 푛 matrices, then

퐵퐴 −1 = 퐴−1퐵−1 • Inverse operation commutes with the transpose operation:

퐴−1 푇 = 퐴푇 −1

• Prove that an 푛 × 푛 matrix is singular if and only if det(A)=0.

13 28-04-2020 References 1. Cormen, Thomas H., Charles E. Leiserson, Ronald L. Rivest, and Clifford Stein. Introduction to algorithms. MIT press, 2009. 2. Cormen, Thomas H., Charles E. Leiserson, Ronald L. Rivest, and Clifford Stein. "Introduction to algorithms second edition." The Knuth-Morris-Pratt Algorithm, year (2001). 3. Seaver, Nick. "Knowing algorithms." (2014): 1441587647177.

14 28-04-2020 Thank You

27-04-2020 15