Menu cs3102: Theory of Computation What does undecidability mean for problems real people (not just CS theorists and Busy Beavers) care about? Class 21: Undecidability in Theory and Practice • Turing-equivalent Grammar • Universal Programming Languages Exam 2: Out at end of class Spring 2010 • “Return-Oriented Programming” today, due Tuesday at 2:01pm University of Virginia • David Evans Problems Computers Can and Cannot Solve

Adapted from Class 1: Models of Computation Unrestricted Grammar

Machine Replacement Grammar Right and left sides of a grammar rule can be any sequence of Finite Automata (Class 2-5) Regular Grammar (A→aB) terminals and nonterminals.

Pushdown Automata (add a Context-free Grammar aXbY → cZd stack) (Classes 6-7) (Classes 7-9) (A→BC )

Turing machine (add an infinite tape) (Classes 14-20) ? How can we prove unrestricted grammars are equivalent to a Turing machine?

Simulation Proof Simulating TM with UG

Show that we can simulate every Unrestricted Grammar with some TM. Fairly easy, but tedious (not shown): design a TM that does the grammar replacements by writing on the tape.

Show that we can simulate every TM with some Unrestricted Grammar. Simulating TM with UG Simulating TM with UG

(No replacement rules with R on left side)

Initial Configuration Models of Computation

Machine Replacement Grammar . . . w0 w1 w2 Finite Automata (Class 2-5) Regular Grammar (A→aB)

q0 Pushdown Automata (add a Context-free Grammar stack) (Classes 6-7) (Classes 7-9) (A→BC )

Turing machine (add an Unrestricted Grammar infinite tape) (Classes 14-20) (Class 21) (α→β )

Which of these are Universal Programming Language Universal Programming Languages? • Definition: a programming language that can BASIC describe every algorithm. Python Fortran C++ • Equivalently: a programming language that x86 Java can simulate every Turing Machine. COBOL Scheme • Equivalently: a programming language in HTML which you can implement a Universal Turing TeX C# PDF Machine. PostScript JavaScript Ruby Proofs • BASIC, C, C++, C#, Fortran, Java, JavaScript, PDF, Why is it impossible for a PostScript, Python, Ruby, Scheme, TeX, etc. are programming language to be universal – Proof: implement a TM simulator in the PL both universal • HTML (before HTML5) is not universal: and resource-constrained ? – Proof: show some algorithm that cannot be implemented in HTML Resource-constrained means it is possible to determine • An infinite loop an upper bound on the resources any program in the – HTML5 might be a universal programming language can consume. language! (Proof is worth challenge bonus.)

Why so many equally powerful programming languages? All universal programming language are equivalent in power: they can all simulate a TM, which can carry out any mechanical algorithm.

high Programming Language Design Space Proliferation of Universal PLs x86 • “Aesthetics” C – Some people like := , others prefer =. Scheme (display “Hello!”) – Some people think whitespace shouldn’t matter (e.g., Java), others think programs should be formatted like Python print ("Hello!") they mean (e.g., Python) – Some people like goto , others like throw . • Expressiveness vs. Simplicity public class HelloWorld { strict typing, Expressiveness public static void main(String[] args) { Java static – Hard to write programs in SUBLEQ System.out.println ("Hello!"); Ada • Expressiveness vs. “Truthiness” } low } Spec# – How much you can say with a little code vs. how likely it is your code means what you think it does more mistake prone less mistake prone “Truthiness” CS 3330 Condensed

Do most x86 programs contain Instruction Pointer (ip) Universal Turing Machines? Memory load Fetch Instruction Return Address Update ip Stack Return Address Execute Instruction

Return Address

Hovav Shacham. The Geometry of Innocent Flesh on the Bone: Execution Stack Return-into-libc without Function Calls (on the x86). CCS 2007. (not like PDA stack) [Paper link]

CS 2150 Really Condensed Sequences of Instructions x86 programs are just sequences of bytes (1 byte = 8 bits = 2 hex characters) inc %ebp movl $0x0f000000, (%edi) xchg %ebp, %eax ret Instructions are encoded using variable-length (1-15 bytes) First byte is opcode that identifies the type of instruction f7 c7 07 00 00 00 0f 95 45 c3 bb ef be ad de MOV $0xDEADBEEF, %eax 5-byte instruction that writes the constant 0xDEADBEEF into register %eax test $0x00000007, %edi setnzb -61(%ebp) c3 RET 1-byte instruction that returns (jumps to the return address that is stored in a location on the stack) eb fe JMP -2 2-byte instruction that moves the instruction pointer back 2 (Example from Shacham’s paper)

“Return-Oriented Programming” Injecting Malicious Code 41 AC 16 FA A0 44 79 8C 2D 43 B3 47 4C 62 69 80 B8 C9 9F BB 34 99 E9 4E 75 D5 A5 64 ... 5F 5A B6 4B 61 A8 B4 AE 27 85 05 0B CA 20 F5 F2 16 C2 B4 8D 54 13 58 93 94 DC 02 16 h 55 B1 AD 57 80 97 BB DA 39 B3 23 7D B3 BD Return Address 1 D2 87 D6 D2 C5 67 8B BE 5F 09 BB B8 F7 EF Return Address 2 int main (void) { return address g 93 15 1E 8F 5E 4C C1 66 C1 1D 82 06 B7 C1 int x = 9; Return Address 3 f 62 96 00 17 F9 CD 82 2F 93 C2 10 5D DD 21 char s[4]; 4D 16 F4 8E 36 7B 7D 91 C7 D3 E1 49 DB A5 Return Address 4 x e FE A4 61 5C 5D E4 8C 8D 6C 33 C3 46 7E 27 F7 88 25 37 F6 F9 B0 E8 B8 42 11 43 4F 6B Return Address 5 gets(s); s[3] d 57 03 56 FB C8 07 4B 9A F7 FC 1D 8D 0D D5 printf ("s is: %s\n“, s); Return Address 6 s[2] 98 38 0B D7 9B 0F 5B 8D A7 CE F5 66 50 5B printf ("x is: %d\n“, x); c 36 82 0F DA 39 16 35 55 6B C7 D0 48 52 89 … s[1] F6 C7 2C 1F EF B7 56 2D F0 1B 39 2F 26 65 } b C3 69 FB 42 6F DD F7 A1 5D 83 C5 07 43 B9 Execution Stack s[0] a E7 BA DF B7 DD 28 5C 62 6F 2F 17 9F D1 51 EC 82 0C 40 7B 51 91 F5 19 31 B7 E0 C7 0B C Program 5A 03 CB 3A 55 82 60 39 85 92 BE 38 C2 DB EA A7 E6 8C 0B B8 0A 53 35 C2 BA 54 1D CF Stack D8 76 B1 DD F2 4E DF 3F C6 FF A7 BF 4B 89 D4 11 2F 3F 4F F7 93 B5 CB 6A AA 01 E1 E6 … Buffer Overflows Code Red > gcc -o bounds bounds.c int main (void) { > bounds int x = 9; abcdefghijkl char s[4]; s is: abcdefghijkl (User input) x is: 9 gets(s); > bounds printf ("s is: %s\n“, s); abcdefghijklm printf ("x is: %d\n“, x); s is: abcdefghijklmn } x is: 1828716553 = 0x6d000009 > bounds abcdefghijkln s is: abcdefghijkln Note: your results may x is: 1845493769 = 0x6e 000009 vary (depending on > bounds machine, compiler, what aaa... [a few thousand characters] else is running, time of crashes shell day, etc.). This is what makes C fun! What does this kind of mistake look like in a popular server?

Defenses “Return-Oriented Programming” 41 AC 16 FA A0 44 79 8C 2D 43 B3 47 4C 62 69 80 B8 C9 9F BB 34 99 E9 4E 75 D5 A5 64 • Use a type-safe programming language (e.g., 5F 5A B6 4B 61 A8 B4 AE 27 85 05 0B CA 20 F5 F2 16 C2 B4 8D 54 13 58 93 94 DC 02 16 bounds checking) 55 B1 AD 57 80 97 BB DA 39 B3 23 7D B3 BD Return Address 1 D2 87 D6 D2 C5 67 8B BE 5F 09 BB B8 F7 EF Return Address 2 93 15 1E 8F 5E 4C C1 66 C1 1D 82 06 B7 C1 62 96 00 17 F9 CD 82 2F 93 C2 10 5D DD 21 Return Address 3 • Write-xor-Execute pages 4D 16 F4 8E 36 7B 7D 91 C7 D3 E1 49 DB A5 Return Address 4 FE A4 61 5C 5D E4 8C 8D 6C 33 C3 46 7E 27 – When the OS loads a page into memory, it is F7 88 25 37 F6 F9 B0 E8 B8 42 11 43 4F 6B Return Address 5 57 03 56 FB C8 07 4B 9A F7 FC 1D 8D 0D D5 Return Address 6 marked as either executable or writable: can’t be 98 38 0B D7 9B 0F 5B 8D A7 CE F5 66 50 5B 36 82 0F DA 39 16 35 55 6B C7 D0 48 52 89 … both F6 C7 2C 1F EF B7 56 2D F0 1B 39 2F 26 65 – Hence: attacker can inject all the code it wants on 69 FB 42 6F DD F7 A1 5D 83 C5 07 43 C3 B9 Execution Stack E7 BA DF B7 DD 28 5C 62 6F 2F 17 9F D1 51 the stack, but can’t jump to it and execute it ECDefeats 82 0C WoX40 7B defense! 51 91 F5Attacker 19 31 can B7 runE0 anyC7 code0B they want by finding a 5ATuring-complete 03 CB 3A 55 82 set 60 of 39“gadgets” 85 92 itBE in 38 your C2 program DB and jumping to them! EA A7 E6 8C 0B B8 0A 53 35 C2 BA 54 1D CF D8 76 B1 DD F2 4E DF 3F C6 FF A7 BF 4B 89 D4 11 2F 3F 4F F7 93 B5 CB 6A AA 01 E1 E6 …

Does it really work? Vulnerability Detection

• Likelihood of finding enough gadgets in Input : an x86 program P “random” bytes to make Turing-complete Output: True if there is some input w, such that libc (C library included in nearly all Unix programs) running P on w allows the attacker to contains more than enough (18MB ~ expect to overwrite return addresses on stack; False have ~ 74000 RET (c3) instructions) otherwise. • Demonstration of attack on voting machine:

http://www.youtube.com/watch?v=lsfG3KPrD1I Example: Morris Internet Worm (1988) Vulnerability Detection P = fingerd Input : an x86 program P – Program used to query user status (running on most Unix servers) Output: True if there is some input w, such that isVulnerable(P)? running P on w allows the attacker to overwrite return addresses on stack; False Yes, for w = “ nop 400 pushl $68732f pushl $6e69622f movl sp,r10 pushl $0 pushl $0 pushl r10 pushl $3 movl otherwise. sp,ap chmk $3b ” – Worm infected several thousand computers (~10% of Internet in 1988)

Vulnerability Detection is Undecidable Vulnerability Detection is Undecidable

“Solving” Undecidable Problems “Impossibility” of Vulnerability Detection

• Undecidable means there is no program that 1. Always gives the correct answer, and 2. Always terminates • Must give up one of these: – Giving up #2 is not acceptable in most cases – Must give up #1: cannot be correct on all inputs • Or change the problem – e.g., modify P to make it invulnerable, etc. Actual Vulnerability Detectors

• Sometimes give the wrong answer: Computability in – “False positive”: say P is a vulnerable when it isn’t Theory and Practice – “False negative”: say P is safe when it is • Heuristics to find common errors • Heuristics to rank-order possible problems (Intellectual Computability Can Microsoft squash 63,000 bugs in Windows 2000? Discussion on TV) … Overall, there are more than 65,000 "potential issues" that could emerge as problems, as discovered by Microsoft's Prefix tool. Microsoft is estimating that 28,000 of these are likely to be "real" problems. http://video.google.com/videoplay?docid=1623254076490030585#

Ali G Problem Input: a list of numbers (mostly 9s) Output: the product of the numbers

LALIG = { < k0, k1, …, kn, p> | each ki represents a number and p represents a

number that is the product of all the kis. numbers }

Yes. It is easy to see a simple algorithm Is LALIG decidable? (e.g., elementary school multiplication) that decides it. Can real computers solve it?

Ali G was Right!

• Theory assumes ideal computers: – Unlimited, perfect memory – Unlimited (finite) time • Real computers have: – Limited memory, time, power outages, flaky programming languages, etc. – There are many decidable problems we cannot solve with real computer: the actual inputs do matter (in practice, but not in theory!) Charge

• Exam 2 out now • Due at beginning of class, Tuesday

• It has some pretty tough questions (and no really easy questions): don’t get stressed out if you can’t answer everything