Probability and Random Processes ECS 315

Asst. Prof. Dr. Prapun Suksompong [email protected] 4 Combinatorics

Office Hours: BKD, 6th floor of Sirindhralai building Tuesday 9:00-10:00 Wednesday 14:20-15:20

1 Thursday 9:00-10:00 Supplementary References  Mathematics of Choice How to count without counting  By Ivan Niven  permutations, combinations, binomial coefficients, the inclusion-exclusion principle, combinatorial probability, partitions of numbers, generating polynomials, the pigeonhole principle, and much more  A Course in Combinatorics  By J. H. van Lint and R. M. Wilson

2 Cartesian product

 The Cartesian product A × B is the set of all ordered pairs where and .

 Named after René Descartes

His best known philosophical statement is “Cogito ergo sum” (French: Je pense, donc je suis; I think, therefore I am)

3 [http://mathemagicalworld.weebly.com/rene-descartes.html] Heads, Bodies and Legs flip-book

4 Heads, Bodies and Legs flip-book (2)

5 Interactive flipbook: Miximal

6 [http://www.fastcodesign.com/3028287/miximal-is-the-new-worlds-cutest-ipad-app] One Hundred Thousand Billion Poems

7 [https://www.youtube.com/watch?v=2NhFoSFNQMQ] One Hundred Thousand Billion Poems  Cent mille milliards de poèmes

8 Pokemon Go: Designing how your avatar looks

9 How many chess games are possible?

10 [https://www.youtube.com/watch?v=Km024eldY1A] An Estimate: The Shannon Number

11 [https://www.youtube.com/watch?v=Km024eldY1A] An Estimate: The Shannon Number

12 [https://www.youtube.com/watch?v=Km024eldY1A] An Estimate: The Shannon Number

13 [https://www.youtube.com/watch?v=Km024eldY1A] The Shannon Number: Just an Estimate

14 [https://www.youtube.com/watch?v=Km024eldY1A] Example: Sock It Two Me

15 [Greenes, 1977] Example: Sock It Two Me

 Jack is so busy that he's always throwing his socks into his top drawer without pairing them. One morning Jack oversleeps. In his haste to get ready for school, (and still a bit sleepy), he reaches into his drawer and pulls out 2 socks.  Jack knows that 4 blue socks, 3 green socks, and 2 tan socks are in his drawer. 1. What are Jack's chances that he pulls out 2 blue socks to match his blue slacks? 2. What are the chances that he pulls out a pair of matching socks? 16 “Origin” of Probability Theory  Probability theory was originally inspired by gambling problems. [http://www.youtube.com/watch?v=MrVD4q1m1Vo]  In 1654, Chevalier de Mere invented a gambling system which bet even money on case B.  When he began losing money, he asked his friend Blaise Pascal to analyze his gambling system.  Pascal discovered that the Chevalier's system would lose about 51 percent of the time.  Pascal became so interested in probability and together with another famous mathematician, Pierre de Fermat, they laid the foundation of probability theory.

17 best known for Fermat's Last Theorem Example: The Seven Card Hustle  Take five red cards and two black cards from a pack.  Ask your friend to shuffle them and then, without looking at the faces, lay them out in a row.

 Bet that them can’t turn over three red cards.  The probability that they CAN do it is

5 5 43 2 3   7 7 3 76 5 5  3 5! 3!4! 1 2    543  7 3!2! 7! 7 6 5 7 3 [Lovell, 2006] 18  Permutation

19 [https://www.youtube.com/watch?v=2EN6jYMiGF4] Permutation  There are 11! = 39,916,800 ways to permute 11 persons.

20 [https://www.youtube.com/watch?v=2EN6jYMiGF4] Finger-Smudge on Touch-Screen Devices  Fingers’ oily smear on the screen  Different apps gives different finger-smudges.  Latent smudges may be usable to infer recently and frequently touched areas of the screen--a form of information leakage.

[http://www.ijsmblog.com/2011/02/ipad-finger-smudge-art.html] 21 For sale… Andre Woolery Art

22 [http://www.andrewooleryart.com/all-artwork/] Andre Woolery Art Fruit Ninja Facebook

23 Angry Bird Mail Lockscreen PIN / Passcode

[http://lifehacker.com/5813533/why-you-should-repeat-one-digit-in-your-phones-4+digit-lockscreen-pin] 24 Smudge Attack  Touchscreen smudge may give away your password/passcode  Four distinct fingerprints reveals the four numbers used for passcode lock.

25 [http://www.engadget.com/2010/08/16/shocker-touchscreen-smudge-may-give-away-your-android-password/2] Suggestion: Repeat One Digit  Unknown numbers:  The number of 4-digit different passcodes = 104  Exactly four different but known numbers:  The number of 4-digit different passcodes = 4! = 24  Exactly three different but known numbers: 50% improvement 34  36  The number of 4-digit different passcodes =  2

Choose the Choose the number that locations of will be the two non- repeated repeated numbers. 26 News: Most Common Lockscreen PINs  Passcodes of users of Big Brother Camera Security iPhone app  15% of all passcode sets were represented by only 10 different passcodes

out of 204,508 recorded passcodes 27 [http://amitay.us/blog/files/most_common_iphone_passcodes.php (2011)] Even easier in Splinter Cell  Decipher the keypad's code by the heat left on the buttons.  Here's the keypad viewed with your thermal goggles. (Numbers added for emphasis.) Again, the stronger the signature, the more recent the keypress.  The code is 1456.

28 Actual Research  University of California San Diego  The researchers have shown that codes can be easily discerned from quite a distance (at least seven metres away) and image- analysis software can automatically find the correct code in more than half of cases even one minute after the code has been entered.  This figure rose to more than eighty percent if the thermal camera was used immediately after the code was entered.

K. Mowery, S. Meiklejohn, and S. Savage. 2011. “Heat of the Moment: Characterizing the Efficacy of Thermal-Camera Based Attacks”. Proceed- ings of WOOT 2011. http://cseweb.ucsd.edu/~kmowery/papers/thermal.pdf http://wordpress.mrreid.org/2011/08/27/hacking-pin-pads-using- thermal-vision/

29 Alternative Solution  Scramble PIN Layout in CyanogenMod (a modified version of Android )

Scramble the keypad when you unlock your phone so that people can't peek at your 30 keystrokes and learn your PIN. The Birthday Problem (Paradox)  How many people do you need to assemble before the probability is greater than 1/2 that some two of them have the same birthday (month and day)?  Birthdays consist of a month and a day with no year attached.  Ignore February 29 which only comes in leap years  Assume that every day is as likely as any other to be someone’s birthday  In a group of r people, what is the probability that two or more people have the same birthday?

31 The Birthday Problem (con’t)  With 88 people, the probability is greater than 1/2 of having three people with the same birthday.  187 people gives a probability greater than1/2 of four people having the same birthday

[Rosenhouse, 2009, p 7] [E. H. McKinney, “Generalized Birthday Problem”: American Mathematical Monthly, Vol. 73, No.4, 1966, pp. 385-87.]

32 Birthday Coincidence: 2nd Version  How many people do you need to assemble before the probability is greater than 1/2 that at least one of them have the same birthday (month and day) as you?

 In a group of r people, what is the probability that at least one of them have the same birthday (month and day) as you?

33 Permuting four distinct letters: A,B,C,D

DCBA CDBA BCDA ACBD DCAB CDAB BCAD ACDB DBCA CBDA BDCA ABCD DBAC CBAD BDAC ABDC DABC CABD BADC ADBC DACB CADB BACD ADCB

34 D Permuting four letters: A,A,B,C

ACBA CABA BCAA ACBA ACAB CAAB BCAA ACAB ABCA CBAA BACA ABCA ABAC CBAA BAAC ABAC AABC CABA BAAC AABC AACB CAAB BACA AACB

35 D Permuting four letters: A,A,B,C

ACBA CABA BCAA ACBA ACAB CAAB BCAA ACAB ABCA CBAA BACA ABCA ABAC CBAA BAAC ABAC AABC CABA BAAC AABC AACB CAAB BACA AACB

36 D Permuting four letters: A,A,B,C

DCBA CDBA BCDA ACBD DCAB CDAB BCAD ACDB DBCA CBDA BDCA ABCD DBAC CBAD BDAC ABDC DABC CABD BADC ADBC DACB CADB BACD ADCB

37 D Permuting four letters: A,A,B,C Group 1 DCBA CDBA BCDA ACBD DCAB CDAB BCAD ACDB DBCA CBDA BDCA ABCD DBAC CBAD BDAC ABDC DABC CABD BADC ADBC DACB CADB BACD ADCB

Group 2

38 Distinct Passcodes (revisit)  Unknown numbers:  The number of 4-digit different passcodes = 104  Exactly four different numbers:  The number of 4-digit different passcodes = 4! = 24  Exactly three different numbers: 34  36  The number of 4-digit different passcodes =  2  Exactly two different numbers: 4 4 4  14 The number of 4-digit different passcodes = 3 2 1  Exactly one number:  The number of 4-digit different passcodes = 1  Check: 10 10 10 10 ⋅24 ⋅36 ⋅14 ⋅ 1 10,000 4 3 2 1 39 Need more practice? [ http://en.wikipedia.org/wiki/Poker_probability ] Ex: Poker Probability

40 [http://www.wolframalpha.com/input/?i=probability+of+royal+flush] Ex: Poker Probability

41 [http://www.wolframalpha.com/input/?i=probability+of+full+house] Ex: Poker Probability

42 [http://www.wolframalpha.com/input/?i=probability+3+queens+2+jacks&lk=3] Ex: Poker Probability

43 Binomial Theorem

()()xy11 x 2y 2

 x12xx 12yy 12x yy 12

()()()xy11 x 2y 2x 33y

 x123xx xx 123y x 123y xx 123yy y 123xx y 123x yyy 123x yyy 123

xxx123 x

y123 y y y

()()xy x y  xxxy yxyy x2 2 xy y2 ()()()x  yxyxy  xxx xxy xyx xyyyxx yxyyyx yyy x3 3xy2 3xy2 y3 44 How many chess games are possible?: A Revisit

53 [https://www.youtube.com/watch?v=Km024eldY1A] Sequence A048987

0 1  Number of possible chess games at the end of (16 pawn moves 1 20 and 4 knight moves) the n-th ply (move). 2 400  Often called perft(n) 3 8902  The name “perft” comes from a command in the chess 4 197281 playing program Crafty. 5 4865609  https://oeis.org/A048987 6 119060324 7 3195901860 8 84998978956 9 2439530234167 10 69352859712417 11 2097651003696806 12 62854969236701747 13 1981066775000396239

54 [http://wismuth.com/chess/statistics-games.html] It all began at Cornell…

55 [https://www.youtube.com/watch?v=vKPRRUWbWAQ] [http://www.ieee.org/documents/hamming_rl.pdf] IEEE Richard W. Hamming Medal

1988 - Richard W. Hamming 2002 - 1989 - Irving S. Reed 2003 - and 1990 - Dennis M. Ritchie and Kenneth L. 2004 - Jack K. Wolf Thompson 2005 - Neil J.A. Sloane 1991 - Elwyn R. Berlekamp 2006 -Vladimir I. Levenshtein 1992 - Lotfi A. Zadeh 2007 - 1993 - Jorma J. Rissanen 2008 - Sergio Verdú 1994 - 2009 - Peter Franaszek 1995 - 2010 -, 1996 - Mark S. Pinsker and 1997 -Thomas M. Cover 2011 - 1998 - David D. Clark 2012 - , Amin Shokrollahi 1999 - David A. Huffman 2013 - Arthur 2000 - Solomon W. Golomb 2014 -Thomas Richardson and Rüdiger L. 2001 - A. G. Fraser Urbanke For contributions to coding theory and its 56 applications to communications, computer science, mathematics and statistics. ences] Neil J.A. Sloane  British-U.S. mathematician.  Received his Ph.D. in 1967 at .  Joined AT&T in 1968 and retired from AT&T Labs in 2012. ane-the-man-who-loved-only-integer-sequ  Major contributions are in the fields of combinatorics, error- correcting codes, and sphere packing.  Best known for being the creator and maintainer of the On-Line Encyclopedia of Integer Sequences. s-adventures-in-numberland/2014/oct/07/neil-slo

57 [https://www.theguardian.com/science/alex How good was the Shannon’s estimation?

20 10 Actual number of possible games Shannon's estimates

15 10

10 10

5 10

0 10 0 2 4 6 8 10 12 14 58 #moves It is quite easy to get a better approximation

20 10 Actual number of possible games Shannon's estimates Prapun's estimates 15 10

10 10

5 10

0 10 0 2 4 6 8 10 12 14 59 #moves