DD2458, Problem Solving and Programming Under Pressure Lecture 9: Arithmetics II

Date: 2008-11-10 Scribe(s): Marcus Forsell Stahre and David Schlyter Lecturer: Douglas Wikström

This lecture is a continuation of the previous one, and covers modular arithmetic and topics from a branch of known as elementary number theory. Also, some abstract algebra will be discussed.

1

Definition 1.1 If an integer d divides another integer n with no remainder, d is said to be a divisor of n. That is, there exists an integer a such that a · d = n. The notation for this is d | n.

Definition 1.2 A common divisor of two non-zero integers m and n is a positive integer d, such that d | m and d | n.

Definition 1.3 The Greatest Common Divisor (GCD) of two positive integers m and n is a common divisor d such that every other common divisor d0 | d. The notation for this is GCD(m, n) = d.

That is, the GCD of two numbers is the greatest number that is a divisor of both of them. To get an intuition of what GCD is, let’s have a look at this example.

Example Calculate GCD(9, 6). Say we have 9 black and 6 white blocks. We want to put the blocks in boxes, but every box has to be the same size, and can only hold blocks of the same color. Also, all the boxes must be full and as large as possible . Let’s for example say we choose a box of size 2:

As we can see, the last box of black bricks is not full. It’s easy to see that it wouldn’t be, since 2 isn’t a divisor of 9. We know that 3 is a divisor of both 9 and 6, so we try this size of boxes.

1 2 DD2458 – Popup HT 2008

There is no larger number that is a divisor of both 9 and 6, so there can’t be any larger size of boxes that can be completely filled. We therefore come to the conclusion that GCD(9, 6) must be 3. From the third definition above, it’s not very hard to see that the Greatest Common Divisor is equivalent to the greatest common factor (not to be confused with prime factors!), since every other common divisor is smaller than GCD(m, n). The third definition also gives us an inefficient but simpler way than the above to find the Greatest Common Divisor of two numbers. First, find the prime factorizations of m and n. Take all the common prime factors and calculate the product of them. Let’s call this product d. Since both m and n is divisible by this d, and since every common divisor is a divisor of d, we get GCD(m, n) = d. Note that we know that every common divisor is a divisor of d since every common divisor can be written as a product of common prime factors.

Example Calculate GCD(60, 42)

60 = 2 · 2 · 3 · 5 42 = 2 · 3 · 7

GCD(60, 42) = 2 · 3 = 6

Since finding the prime factorization of a number in general is hard, this is obviously not a very good way to calculate GCD, especially when m and n grows larger. There are much more efficient ways to calculate it, and we will talk about two of them in a moment, but first let’s have a look at a few useful properties of GCD.

1.1 Properties of GCD m and n are positive integers, and k is any integer.

GCD(m, n) = GCD(n, m) (1) GCD(m, n) = GCD(m + k · n, n) (2) GCD(m, n) = GCD(m mod n, n) (3) GCD(m, n) = 2 · GCD(m/2, n/2) if m and n are even. (4) GCD(m, n) = GCD(m/2, n) if m is even and n is odd. (5)

It is not hard too see that (1) is true, but the others might need some explanation. Property (2) is the basis for (3), so let’s start with (2).

If x is a common divisor of m and n, then x | m and x | n. Therefore x | k · n, and thus x | (m + k · n). Arithmetics II 3

If x is a common divisor of (m + k · n) and n, then x | m + k · n and x | n. Therefore x | k · n, and thus x | (m + k · n) − k · n ⇔ x | m.

From this we conclude that the set of common divisors to m and n is equal to the set of common divisors to m + k · n and n. This means that GCD(m, n) = GCD(m + k · n, n)

We’ll now use this to show (3). We are going to use the fact that m mod n is defined as an integer 0 ≤ r < n such that for some integer a, m = a · n + r. From this we get r = m − a · n, and from (2) we know that GCD(m − a · n, n) = GCD(m, n).

Since m and n both being even is equivalent to m and n having a common di- visor 2, we can move this factor outside of the GCD function, and still get the same result (4). By similar reasoning, if one of m and n are even and not the other one, 2 is a non-common divisor, and the result is not affected if it’s removed (5).

1.2 Euclidean

The makes use of property (3) from above. The idea is to reduce m and n repeatedly until one of them is zero. The original version of the algorithm actually used property (2) from above, and looks like this:

Algorithm 1: Euclidean algorithm for finding the GCD Input: Two integers, m and n Output: The GCD of m and n EuclideanOld(m,n) (1) if m = 0 (2) return n (3) while n 6= 0 (4) if n > m (5) n ← n − m (6) else (7) m ← m − n (8) return m

It uses the fact that GCD(m, n) = GCD(m − n, n), and simply repeats the op- eration until n hits zero. A more efficient way of doing this is of course to use (3). 4 DD2458 – Popup HT 2008

Algorithm 2: A better version of the Euclidean algorithm Input: Two integers, m and n Output: The GCD of m and n EuclideanIterative(m,n) (1) while n 6= 0 (2) t ← n (3) n ← m mod n (4) m ← t (5) return m

This is very much the same idea as the previous one. GCD(m, n) = GCD(n, m mod n), repeat until n hits zero. Since modulo basically is repeated subtraction, this is very much the same algorithm, but several subtractions are “done at once”. The recursive version of the algorithm looks like this:

Algorithm 3: A recursive version of Algorithm 2 Input: Two integers, m and n Output: The GCD of m and n EuclideanRecursive(m,n) (1) if n = 0 (2) return m (3) else (4) return EuclideanRecursive(n, m mod n)

1.3 Binary GCD Algorithm

When using a computer to calculate GCD, the Binary GCD Algorithm (also known as Stein’s algorithm) might be preferable, since it makes good use of the binary representation of numbers that a computer uses. The idea is to use property (4) and (5) in order to step by step decrease the number of bits in the numbers. Arithmetics II 5

Algorithm 4: Stein’s algorithm for binary GCD calculation Input: Two integers, m and n Output: The GCD of m and n BinaryGCD(m,n) (1) if m = 0 or n = 0 (2) return 0 (3) s ← 0 (4) while m and n are even (5) m ← m/2 (6) n ← n/2 (7) s ← s + 1 (8) while n is even (9) n ← n/2 (10) while m 6= 0 (11) while m is even (12) m ← m/2 (13) if m < n (14) Swap(m,n) (15) m ← m − n (16) m ← m/2 (17) return 2s · n

First, all the common factors 2 are removed. Since the numbers are stored in binary on a computer, this can be done with shift-operations, which are very fast. If n still has any factors 2, they can be discarded, as they’re not common factors. The main-loop of the algorithm is then entered. At this point there are no common factors 2 that are left, since they’ve all been removed. Therefore, m can be divided by 2 until it is odd. If it then is smaller than n, they are swapped. This is done so that the next operation never gives a negative result. When row 15 is about to be executed, both m and n are guaranteed to be odd numbers. And since the difference of two odd numbers always is even, row 15 always makes m into an even number. It can therefore be divided by two once again. The loop with shifting right and making even continues until m is zero, and then n is returned, but not before multiplying with the common factors 2 that were removed on rows 4 to 7.

2 Bézout’s identity

Bézout’s identity states that for every pair of non-zero integers a and b, there exists integers x and y such that ax + by = GCD(a, b). These can be found using the Extended Euclidean Algorithm. Note though that there is an infinite number of x and y that satisfies this equation. The Extended Euclidean Algorithm finds one solution, and every other solution is on the following form, where k is an integer:

b xk = x + k · GCD(a,b) a yk = y − k · GCD(a,b) 6 DD2458 – Popup HT 2008

Later on, there will be an example of when this identity and the Extended Euclidean Algorithm can be useful.

2.1 Extended Euclidean Algorithm

Algorithm 5: Extended Euclidean algorithm for solving ax + by = GCD(a,b) Input: Two integers, m and n Output: Two integers, x and y satisfying the equation ExtendedEuclidean(m,n) (1) if n | m (2) return (0, 1) (3) else (4) (x0, y0) ← ExtendedEuclidean(n, m mod n) 0 0 j m k (5) return (y’, x − y · n )

If n is a divisor of m, then the solution to mx + ny = GCD(m, n) must be x = 0, y = 1, since we know then that GCD(m, n) is n. In the recursive step we use GCD property (3) from above and instead try to solve the equation n · x0 + 0 m 0 (m mod n) · y = GCD(m, n). By introducing a term n · b n c · y , we can use the m fact that n · b n c + (m mod n) = m (This is simply integer division followed by multiplication, and then adding back the missing remainder). n · x0 + (m mod n) · y0 = GCD(m, n) jmk jmk n · x0 + (m mod n) · y0 + n · · y0 − n · · y0 = GCD(m, n) n n jmk jmk n · (x0 − · y0) + ((m mod n) + n · ) · y0 = GCD(m, n) n n jmk n · (x0 − · y0) + m · y0 = GCD(m, n) n This means that the solution to the equation m · x + n · y = GCD(m, n), by the 0 0 m 0 recursion, is x = y , and y = (x − b n c · y ).

3 Abstract Algebra

Abstract algebra is a branch of mathematics that deals with algebraic structures such as rings, groups, fields, and vector spaces. When dealing with number theory these structures are very often useful, so this is a brief introduction to a few of them.

3.1 Monoids A monoid consists of a set M and a binary operator ◦, such that:

For every a, b ∈ M: a ◦ b ∈ M (Closure) For every a, b, c ∈ M: (a ◦ b) ◦ c = a ◦ (b ◦ c) (Associativity) There is an e ∈ M such that for every a ∈ M: a ◦ e = e ◦ a = a (Identity) Arithmetics II 7

Note that you could have a structure without the identity property, but it wouldn’t be a monoid, and you’d end up with very strange results! If the monoid also has the property that for every a, b ∈ M: a ◦ b = b ◦ a, the monoid is said to be commutative, or abelian.

3.2 Groups A group is a monoid (G, ∗) such that:

For every a ∈ G there is an b ∈ G such that: a · b = b · a = e (Inverse)

A group does not need to be abelian, but it might be. It’s often natural to denote the binary operation on a group as addition and multiplication, and in those cases the group is usually referred to as an additive or multiplicative group, respectively. In those cases it’s also often natural to have 0 or 1 as the identity element.

3.3 Application of groups Exponentiation is in general the same thing as repeated multiplication, just as multiplication is repeated addition. We can therefore in a multiplicative group do exponentiation:

Example We have a group (Z, ∗). Find 23

Calculating 23 is the same as multiplying three times, so we get 2·2·2. 2 · 2 · 2 = 4 · 2 = 8 Therefore 23 = 8

When dealing with larger exponents, this way of doing it will be very slow. We can instead use a simple method called Square-and-Multiply (or Double-and-Add when dealing with an additive group). The following algorithm calculates gx. Algorithm 6: Square and Multiply Input: A base g and an exponent x Output: The result of gx SquareAndMultiply(g, x) (1) r ← 1 (2) while x > 0 (3) if x is odd (4) r ← r · g (5) g ← g · g (6) x ← Floor(x/2) (7) return r

Let g be a number, which in base 2 is represented by the binary digits an−1an−2...a0, x where a0 is the least significant digit. We can then calculate g as

Y ak g2 (6)

k:ak=1 8 DD2458 – Popup HT 2008

0 1 n−1 This algorithm uses the fact that g = a0 · 2 + a1 · 2 + ... + an−1 · 2 . That is, if we go through all the binary digits, and the k’th binary digit is a one, multiply the k k result by g2 . In the implementation this g2 is kept between the digits, and only one additional multiplication needs to be do for each bit in x. This means that the running time of this algorithm is roughly O(n3) (since multiplication usually takes n2 time). It’s important to notice that the binary operator doesn’t need to be multi- plication, it might just as well have been addition or anything else. This quite simple algorithm is not only useful in simple groups, but can also be used to do fast exponentiation with positive integers.

3.4 Ring A commutative ring (R, +, ·) is a set R and two binary operators + and · such that: (R, +) is an abelian group with identity 0 (R, ·) is an abelian monoid with identity 1 For every a, b, c ∈ R: a · (b + c) = a · b + a · c, (a + b) · c = a · c + b · c

3.5 Field A field is a ring where multiplication is commutative, and where every element except zero has a multiplicative inverse. Examples of fields are R and Zp (where p is a prime). Fields are useful because they often behave just like regular numbers. If (K, +, ·) is a field, then

(K − {0}, ·) is an abelian group (Multiplicative Inverse)

4 Modular Arithmetics

Modular arithmetics, is very often useful when dealing with cyclic structures like groups and rings, and they are also often useful in computer science when dealing with for example cryptography. It basically deals with arithmetics with numbers that “wrap around” at some value. In general, modular arithmetics can be consid- ered to be arithmetics modulo some integer n.

Theorem 4.1 Given two integers n and m, there exists an integer q and an integer r such that n = qm + r. The notation for this is n mod m = r.

n When n and m are positive integers, we have that n = b m c + (n mod m) (We used this above when showing the correctness of the Extended Euclidean Algorithm).

Definition 4.2 Two integers n and n0 are congruent modulo m if and only if n = n0 + k · m for some integer k, that is, n mod m = n0 mod m. The notation for this is n ≡ n0 (mod m)).

0 0 0 Other common notations for this is n ≡ n mod m, n = n (mod m), n ≡m n , and n ≡ n0(m). Arithmetics II 9

It’s also worth to note that the congruence relation is an equivalence relation. This means that it has the following properties:

a ≡ a (Reflexive) if a ≡ b then b ≡ a (Symmetric) if a ≡ b and b ≡ c then a ≡ c (Transitive) The following theorems might also be useful to know:

Theorem 4.3 The set 0, 1, ..., n − 1 with modular addition and multiplication forms a commutative ring. It can also be defined as the set of elements in the equivalence classes under the congruence relation modulo n.

Common notations: Zn, Z/nZ, Z/(n), Z/n, Z/(n)Z

Theorem 4.4 If n is prime, Zn is a field.

5 Useful Theorems

Two integers are said to be relatively prime, or co-prime, if they don’t have any common prime factors. That is:

Theorem 5.1 Two integers m and n are co-prime if their greatest common divisor is 1.

5.1 The Chinese Remainder Theorem The Chinese Remainder Theorem has its origins in a Chinese book called “The Mathematical Classics by Sun Zi”, and is a statement about simultaneous congru- ences. It is said to have it origins from when the emperor was to count how many soldiers he had. They ordered the soldiers to stand in lines of for example 5, and then counted how many was left over. This was repeated a few times with different line-widths, and at the end they could calculate how many soldiers there were.

Claim 5.2 Let n1, n2, ..., nk be pairwise relatively prime integers, and let a1, a2, ..., ak be integers. Then the following system of equations has a solution: x ≡ a1 (mod n1) x ≡ a2 (mod n2) x ≡ a3 (mod n3) ... x ≡ ak (mod nk)

The constructive proof of this, which is also basis for the implementation of the algorithm, is as follows:

(1) Let N = n1 · n2 · n3 · ... · nk N (2) Find an ri and si such that ri · ni + si · = 1 ni 10 DD2458 – Popup HT 2008

( N 1 mod ni (3) Note that: si · = 1 − ri · ni = ni 0 mod nj if j 6= i k X N (4) The solution is found by x = s · · a i n i i=1 i

The key to the proof is in step 3. In step two we find the solution (ri, si) to the equation using the extended euclidean algorithm. Also take notice that ni and N are co-prime. In step 3 we have just rewritten the equation from step 2. It’s ni N easy to see that si · ≡ 1 (mod ni), because ri · ni ≡ 0 (mod ni), and since N ni N has every factor nj except ni, it’s easy to see that si · ≡ 0 (mod nj). This fact ni can be used to prove that (4) is a solution modulo N, but since the proof of this is quite long and tedious, it will not be presented in this text.

5.2 Fermat’s little theorem This theorem is also very useful when dealing with number theory, and is for ex- ample the basis of the Fermat .

Theorem 5.3 Let p be any prime. ap ≡ a (mod p) for every integer a.

Theorem 5.4 Let p be any prime, and a an integer co-prime to p, then ap−1 ≡ 1 (mod p)

Both of the above statements are equivalent. There are several proofs for the correctness of them, but one in particular is very short and easy to understand. The basic idea is to make bracelets of length p using a different colors. Every bracelet using only one kind of color is removed (this is exactly a bracelets), and the proof continues by showing that the remaining number of bracelets is divisible by p.

5.3 Euler’s totient function Euler’s totient function, or Euler-phi is defined as follows:

Definition 5.5 φ(n) is the number of positive integers less than n that are rela- tively prime to n.

It’s not hard to see that φ(p) = p−1 for every prime p. The formula for calculating the value of the function in the general case is not hard either. Consider the prime factorization of n:

n = p1 · p2 · p3 · ... · pk

every p1’th number smaller than n is divisible by p1. every p2’th number smaller than n is divisible by p2. ... every pk’th number smaller than n is divisible by pk.

Therefore, φ(n) = n · (1 − 1/p1) · (1 − 1/p2) · ... · (1 − 1/pk). Arithmetics II 11

In essence, we’re simply removing all the numbers that aren’t co-prime to n. The Y 1 above is often written as an Euler product: n · (1 − ) p p|n

5.4 Modular multiplicative inverse There are several ways to calculate the inverse of an element in group. We will now show two ways to do it - one using Square-and-Multiply, and one using Extended Euclidean.

To start with, we will show yet one example of how the Bézout identity can be useful. A modular multiplicative inverse to a number n only exists if n and p are co-prime, that is GCD(n, p) = 1. That means the Extended Euclidean Algorithm solves the equation nx + py = 1, which modulo p is the same as nx ≡ 1 (mod p). That is, the x that the Extended Euclidean Algorithm calculates is the inverse to n in the integer ring modulo p.

We know that for a modular inverse to exists, GCD(n, p) = 1, so p must be co- prime to all n. This means that p is a prime, and we can make use of Fermat’s theorem. Since we get from Fermat that np−1 ≡ 1 (mod p), we easily see that n · np−2 ≡ 1 (mod p), which means that the inverse of n is np−2, and we can easily calculate np−2 by Square-and-Multiply.

5.5 Vandermonde Matrix Theorem 5.6 A Vandermonde matrix is always invertible

A Vandermonde matrix is a matrix with the following structure:

 2 n−1 1 α1 α1 ··· α1 2 n−1 1 α2 α2 ··· α2   2 n−1 1 α3 α ··· α   3 3  (7) ......  . . . . .  2 n−1 1 αm αm ··· αm

If you let A be a Vandermonde matrix, and solve a system of equations Au = y, Pn−1 i you will actually find the coefficients of the polynomial P (x) = i=0 ui · x . In this case, yi should be the y-value of the function at the point αi.