I 3YSTEMSAND)NTERNET I )NFRASTRUCTURE3ECURITY

.ETWORKAND3ECURITY2ESEARCH#ENTER $EPARTMENTOF#OMPUTER3CIENCEAND%NGINEERING 0ENNSYLVANIA3TATE5NIVERSITY 5NIVERSITY0ARK0!

CSE543 - Introduction to and Network Security Module: Security

Professor Patrick McDaniel Fall 2008

CSE543 - Introduction to Computer and Network Security 1 OS Security • An secure OS should provide (at least) the following mechanisms ‣ ‣ File protection ‣ General object protection ‣ Access authentication • How do we go about designing a trusted OS? • “Trust” in this context means something different from “Secure”

CSE543 - Introduction to Computer and Network Security Page 2 Trust vs. Security • When you get your medication at a pharmacy, you are “trusting” that it is appropriate for the condition you are addressing. In effect, you are arguing internally: ‣ The doctor was correct in prescribing this drug ‣ The FDA vetted the drug through scientific analysis and clinical trials ‣ No maniac has tampered with the bottle • The first two are are matters “trust”, and the last is a matter of “security” • An OS needs to perform similar due diligence to achieve “trust” and “security”

CSE543 - Introduction to Computer and Network Security Page 3 Access Control Lists • ACL: a list of the principals that are authorized to have access to some object. • Or more correctly: • Eg., O2

O1: S1

S1 Y O2: S1, S2, S3

O3: S3

S2 Y • We are going to see a lot of

S3 Y examples of these throughout the semester.

CSE543 - Introduction to Computer and Network Security Page 4 ACL in systems • ACLs are typically used to implement discretionary access control • For example: you define the file system ACLs using the chmod utility ….

CSE543 - Introduction to Computer and Network Security Page 5 Discretionary Access • The UNIX filesystem implements discretionary access control through file permissions set by user • The set of objects is the files in the filesystem, ‣ e.g., /etc/passwd • Each file an owner and group (subjects) ‣ The owner is typically the creator of the file, and the entity in control of the access control policy ‣ Note: this can be overridden by the “root” user • There is a additional subject called world, which represents everyone else

CSE543 - Introduction to Computer and Network Security Page 6 UNIX filesystem rights … • There are three rights in the UNIX filesystem ‣ READ - allows the subject () to read the contents of the file. ‣ WRITE - allows the subject (process) to alter the contents of the file. ‣ EXECUTE - allows the subject (process) to execute the contents of the file (e.g., program, executable, …)

• Q: why is execute a right? • Q: does the right to read a program implicitly give you

CSE543 - Introduction to Computer and Network Security Page 7 The UNIX FS access policy • Really, this is a bit string encoding an access matrix • E.g., rwx rwx rwx World Group Owner

• And a policy is encoded as “r”, “w”, “x” if enabled, and “-” if not, e.g, rwxrw--x • Says user can read, write and execute, group can read and write, and world can execute only.

CSE543 - Introduction to Computer and Network Security Page 8 Caveats: UNIX Filesystem • Access is often not really this easy: you need to have certain rights to parent directories to access a file (execute, for example) ‣ The reasons for this are quite esoteric • The preceding policy may appear to be contradictory ‣ A member of the group does not have execute rights, but members of the world do, so … ‣ A user appears to be both allowed and prohibited from executing access ‣ Not really: these policies are monotonic … the absence of a right does not mean they should not get access at all, just that that particular identity (e.g., group member, world) should not be given that right.

CSE543 - Introduction to Computer and Network Security Page 9 Tokens

• Like the UID/GID in a UNIX process ‣ User ‣ Group ‣ Aliases ‣ Privileges (predefined sets of rights)

• May be specific to a domain • Composed into global SID

• Subsequent processes inherit access tokens ‣ Different processes may have different rights

CSE543 - Introduction to Computer and Network Security Page 10 Access Control Entries

• DACL in the security descriptor of an object ‣ List of access control entries (ACEs) • ACE structure (proposed by Swift et al) ‣ Type (grant or deny) ‣ Flags ‣ Object Type: global UID for type (limit ACEs checked) ‣ InheritedObjectType: complex inheritance ‣ Access rights: access mask ‣ Principal SID: principal the ACE applies to • Checking algorithm ‣ ACE matches SID (user, group, alias, etc) ‣ ACE denies access for specified right -- deny ‣ ACE grants access for some rights -- need full coverage

CSE543 - Introduction to Computer and Network Security Page 11 Access Checking with ACEs • Example

CSE543 - Introduction to Computer and Network Security Page 12 Window Vista Integrity • Integrity protection for writing • Defines a series of protection level of increasing protection ‣ untrusted (lowest) ‣ low (Internet) ‣ medium (user) ‣ high (admin) ‣ system ‣ installer (highest) • Semantics: If subject’s (process’s) integrity level dominates the object’s integrity level, then the write is allowed

CSE543 - Introduction to Computer and Network Security Page 13 Vista Integrity • Does Vista Integrity protect the integrity of J’s public key file O2?

O1 O2 O3

J R RW RW

S2 N R RW

S3 N R RW

CSE543 - Introduction to Computer and Network Security Page 14 UID Transition: Setuid

• A special bit in the mode bits • Execute file ‣ Resulting process has the effective (and fs) UID/GID of file owner • Enables a user to escalate ‣ For executing a trusted service • Downside: User defines execution environment ‣ e.g., Environment variables, input arguments, open descriptors, etc. • Service must protect itself or user can gain root access

• All UNIX services involves root processes -- many via setuid

CSE543 - Introduction to Computer and Network Security Page 15 /tmp Vulnerability • creat(pathname, mode) • O_EXCL flag ‣ if file already exists this is an error • Potential attack ‣ Attacker creates file in shared space (/tmp) ‣ Give it a filename used by a higher authority service ‣ Make sure that service has permission to the file ‣ If creat is used without O_EXCL, then can share the file with the higher authority process

CSE543 - Introduction to Computer and Network Security Page 16 Other Vulnerabilities • Objects w/o sufficient control ‣ Windows registry, network • Libraries ‣ Load order permits defined libraries • Executables are everywhere ‣ Web content, Email, Documents (Word) • Labeling is wrong ‣ Mount a new file system; device • Malware can modify your permissions ‣ Inherent to discretionary model

CSE543 - Introduction to Computer and Network Security Page 17 Sandboxing • An execution environment for programs that contains a limited set of rights ‣ A subset of your permissions (meet secrecy and integrity goals) ‣ Cannot be changed by the running program (mandatory)

CSE543 - Introduction to Computer and Network Security Page 18 UNIX Chroot • Create a domain in which a process is confined ‣ Process can only read/write within file system subtree ‣ Applies to all descendant processes ‣ Can carry file descriptors in ‘chroot jail’

CSE543 - Introduction to Computer and Network Security Page 19 Chroot Vulnerability • Unfortunately, chroot can trick its own system ‣ define a passwd file at /etc/passwd ‣ run su • su thinks that this is the real passwd file ‣ gives root access • Use mknod to create device file to access physical memory • Setup requires great care ‣ Never run chroot process as root ‣ Must not be able to get root privileges ‣ No control by chrooted process (user) of contents in jail ‣ Be careful about descriptors, open sockets, IPC that may be available

CSE543 - Introduction to Computer and Network Security Page 20 Process-specific Permissions

• Design the permissions of a process specific to its use

• How do we change the permissions of a process in an ACL system?

CSE543 - Introduction to Computer and Network Security Page 21 Confused Deputy Problem • Imagine a multi-client server ‣ Clients have a different set of objects that they can access • In an ACL system, the server always has access to all the objects ‣ What happens if a client tricks the server into accessing into another client’s objects? ‣ Shouldn’t the server only have access to that client’s objects for its requests?

CSE543 - Introduction to Computer and Network Security Page 22 Capabilities • A capability is the tuple (object, rights) • A capability system implements access control by checking if the process has an appropriate capability ‣ Simple, right? ‣ This is a little like a ticket in the Kerberos system

• Q: Does this eliminate the need for authentication?

CSE543 - Introduction to Computer and Network Security Page 23 Capabilities • A: Well, yes and no … • Capabilities remove the overhead of managing per object rights, but add the overhead of managing capabilities • Moreover, to get any real security, they have to be unforgeable ‣ Hardware tags (to protect capabilities) ‣ Protected address space/registers ‣ Language based techniques • Enforce access restrictions on caps. ‣ Cryptography • Make them unforgeable

CSE543 - Introduction to Computer and Network Security Page 24 Real OS Capabilities

Process Table C List RX A RW X C B Process Z R D W E C ...... D

• The OS kernel manages capabilities in the process table, out of reach of the process • Capabilities added by user requests (that comply with policy)

CSE543 - Introduction to Computer and Network Security Page 25 capability? • Well, what are the requirements? ‣ Authenticity/integrity - do not want malicious process to forge capabilities • Start with the data itself: [object, rights] ‣ Object is typically encoded with identifier, or by some other tag (capabilities are sometimes known as tags) ‣ Rights are often fixed (read, modify, write, execute, etc.) • Now, do what you with any other data (assume the kernel has a secret key k)

E(k, [Oi, r1, r2, … rn]) • What’s wrong with this construction (I got it from the website of one of the experts in the area)?

CSE543 - Introduction to Computer and Network Security Page 26 The right construction • Encryption does not provide authenticity/integrity, it provides confidentiality

[Oi, r1, r2, … rn],HMAC(k, [Oi, r1, r2, … rn])

• So how would you attack the preceding construction?

CSE543 - Introduction to Computer and Network Security Page 27 A (fictional) Capability Example

• We use the “ls -lt” command to view the contents of our home directory in a OS implementing capabilities: ‣ Initially, our shell process has RWX capabilities for our home directory, and RX capabilities for all the directories to the root. ‣ The “ls -lt” command is ()ed, and the shell delegates the directory permissions by giving “ls” the capabilities • Note that the capabilities are _not_ tied to any subject ‣ The “ls -lt” process exercises the rights to read the directories structure all the way down to the local ‣ Of course, the “ls -lt” process now need to obtain read rights to the files (to get their specific meta-information), and obtains them by appealing to the security manager (in kernel) -- the request fulfills the policy, and they are added and exercised ‣ The “ls -lt” uses access rights given to the terminal to write output • Note: there are many ways that the policy can be implemented, rights handed off, etc. We will talk about a couple in the following discussions.

CSE543 - Introduction to Computer and Network Security Page 28 Procedure-Level Protection Domains

• HYDRA ‣ Each procedure defines a new protection domain • Procedure ‣ Code ‣ Data ‣ Capabilities to other objects • Caller-independent • Caller-dependent templates • Local Name Space ‣ Capabilities are bound here ‣ Record of a procedure invocation (procedure instance) • Process ‣ Stack of LNSs

CSE543 - Introduction to Computer and Network Security Page 29 How HYDRA works

Caller-Dep Capabilities Call Callee + Capabilities CallerTemplate LNS Caller Proc Capabilities Data

Kernel

Caller-Dep Capabilities

Create Callee CalleeTemplate LNS LNS Callee Proc Capabilities Data

• Q: Which object defines the protection domain?

CSE543 - Introduction to Computer and Network Security Page 30 Implications of Fine-Grained Protection

• Programmer ‣ Must define templates for procedure ‣ Connect the procedure rights together • Performance Impact

• Q: Do we need to manage rights at this level?

CSE543 - Introduction to Computer and Network Security Page 31 Linden’s Capability View

• Achieve flexible, effective security by ‣ Small protection domains ‣ Extensible set of types • Implies a capability system ‣ Small protection domains with least privilege permissions ‣ Extensible types enable composition of systems reliably ‣ Capabilities can be passed among protection domains and into new subsystems • Protected Procedures ‣ Like HYDRA ‣ Change domain with each procedure invocation ‣ New procedure is a new instance • Protection Domain switch time is key ‣ high in modern processors

CSE543 - Introduction to Computer and Network Security Page 32 Correctness Claim • “It is far more difficult to build a 50,000 line program than 1,000 programs that are each 50 lines long.” ‣ What is your opinion of this? ‣ Is it just the procedure development that is important? • Two problems ‣ Decomposition results in inefficiencies ‣ Interactions between procedures are not captured

CSE543 - Introduction to Computer and Network Security Page 33 Flexibility vs. Security • Small protection domains are desirable because: ‣ Enables solving finer-grained problems ‣ Less rigid protection ‣ Independent accounting ‣ Reliable and redundant security controls ‣ Individual controls are easier to understand

CSE543 - Introduction to Computer and Network Security Page 34 Secure Capability Systems • SCAP ‣ Karger’s extension of the Cambridge CAP system • EROS ‣ Shapiro’s reimplementation of the KeyKOS system

CSE543 - Introduction to Computer and Network Security Page 35 Capabilities and the *-Property

• Capabilities and Lattice Models Don’t Mix • Suppose A is higher secrecy than B ‣ A can read B’s capabilities • Q: Can a Trojan horse running as A write to Obj?

B’s capabilities Read-Write Obj

CSE543 - Introduction to Computer and Network Security Page 36 SCAP *-security • Mediate requests to load capabilities ‣ Must be loaded into a capability cache before use • Enforce MLS requirements on capability load ‣ If subject label dominates capability’s object label, then ‣ Change the capability to read-only • Expensive to test for MLS on every load • For general confinement test against confinement property for every load (uses ACLs!)

CSE543 - Introduction to Computer and Network Security Page 37 EROS *-security • Define weak capabilities ‣ If a weak capability is used to fetch a capability (transitively), then the fetched capability becomes read-only and weak • Assign weak capabilities to higher-secrecy subjects for accessing a lower-secrecy write capability ‣ becomes read-only and weak • No need to test against a policy at runtime ‣ Faster performance is possible • For general confinement use an confined processes or authorized capability sets ‣ Not clear these really worked for general confinement

CSE543 - Introduction to Computer and Network Security Page 38 Capability Management • How’d you get those capabilities? ‣ Stored with program, user ‣ Compare with getting permissions by a process label • How do I get them back? ‣ Once granted, nearly impossible to revoke

CSE543 - Introduction to Computer and Network Security Page 39 EROS Revocation • Defined by Redell ‣ Use a layer of indirection • Revoker capabilities ‣ If you may revoke, create a revoker ‣ The grant capabilities to the revoker ‣ When you delete the revoker, all descendants become invalid Revocable Revoker Object

Not Revocable

CSE543 - Introduction to Computer and Network Security Page 40 SCAP Revocation • Chain the capabilities ‣ “revocation by chaining” • All capabilities to an object are stored in a ring ‣ Can then revoke one ‣ Motivate reassessment of all others ‣ How do I know that I am revoking a particular capability? • Compare with using revoker capabilities ‣ the memory/performance cost ‣ the flexibility of revocation

CSE543 - Introduction to Computer and Network Security Page 41 Result • Generally, the security problems with capability systems can be solved • So, why aren’t cap systems more broadly used? ‣ Capability management is difficult • How do I know what rights to give out in the first place? ‣ Defining and testing confinement is expensive or limiting • Test every grant is expensive (supposed to be lots) • Predefine a safe domain is limiting and counterintuitive • Setup per process is key ‣ For ACLs it is setup per object -- may be less volatility

CSE543 - Introduction to Computer and Network Security Page 42 Mandatory Access Control • Is about administration ‣ Policy is defined and fixed for the system • Users cannot modify policy • More importantly, users’ processes cannot modify policy

• So, what should the policy be?

CSE543 - Introduction to Computer and Network Security Page 43 Security Goals • Secrecy ‣ Do not leak data to unauthorized subjects • Integrity ‣ Do not depend on input from lower integrity subjects • Invocation, inputs, files, etc.

CSE543 - Introduction to Computer and Network Security Page 44 MAC Systems

• Major Effort: ‣ Multiprocessing system -- developed many OS concepts • Including security ‣ Begun in 1965 • Development continued into the mid-70s ‣ Used until 2000 ‣ Initial partners: MIT, , GE/Honeywell ‣ Other innovations: hierarchical filesystems, dynamic linking

• Subsequent proprietary system, SCOMP, became the basis for secure operating systems design

CSE543 - Introduction to Computer and Network Security Page 45 Multics Goals

• Secrecy ‣ Multilevel security • Integrity ‣ Rings of protection • Reference Monitoring ‣ Mediate segment access, ring crossing • Resulting system is considered a high point in secure system design

CSE543 - Introduction to Computer and Network Security Page 46 Protection Rings

• Successively less-privileged “domains” • Modern CPUs support 4 rings ‣ Use 2 mainly: Kernel and user • rings ‣ Ring 0 has kernel ‣ Ring 3 has application code

• Example: Multics (64 rings in theory, 8 in practice)

CSE543 - Introduction to Computer and Network Security Page 47 What Are Protection Rings? • Coarse-grained, Hardware Protection Mechanism • Boundary between Levels of Authority ‣ Most privileged -- ring 0 ‣ Monotonically less privileged above • Fundamental Purpose ‣ Protect system integrity • Protect kernel from services • Protect services from applications • So on...

CSE543 - Introduction to Computer and Network Security Page 48 Intel Rules • Each Memory Segment has a privilege level (ring number)

• The CPU has a Current Protection Level (CPL) ‣ Level of the segment where instructions are being read • Program can read/write in segments of lower level than CPL ‣ kernel can read/write user space ‣ user cannot read/write kernel • why not?

CSE543 - Introduction to Computer and Network Security Page 49 Protection Ring Rules

• Program cannot call code of higher privilege directly ‣ Gate is a special memory Ring 3 address where lower-privilege code can call higher No • Enables OS to control where gate applications call it (system calls) Gate

Ring 0

CSE543 - Introduction to Computer and Network Security Page 50 Multics Interpretation • Kernel resides in ring 0 • Process runs in a ring r 7 --- ‣ Access based on current ring 6 • Process accesses data (segment) 5 ‣ Each data segment has an access a2 bracket: (a1, a2) 4 • a1 <= a2 R-X ‣ Describes read and write access to Ring 3 segment 2 • r is the current ring • r <= a1: access permitted 1 a1 • a1 < r <= a2: r and x permitted; w denied RWX • a2 < r: all access denied 0

CSE543 - Introduction to Computer and Network Security Page 51 Multics Interpretation (con’t)

• Also different procedure segments Denied ‣ with call brackets: (c1, c2), c1 <= c2 7 c ‣ and access brackets (a1, a2) 2 6 Allow ‣ The following must be true (a2 == c1) with ‣ Rights to execute code in a new procedure segment 5 gate

• r < a1: access permitted with ring-crossing fault a2 • a1 <= r <= a2 = c1: access permitted and no fault 4 c1 • a2 < r <= c2: access permitted through a valid gate • c2 < r: access denied Ring 3 No ring • What’s it mean? fault 2 ‣ case 1: ring-crossing fault changes procedure’s ring • increases from r to a1 1 a1 ‣ case 2: keep same ring number Ring ‣ case 3: gate checks args, decreases ring number 0 fault • Target code segment defines the new ring

CSE543 - Introduction to Computer and Network Security Page 52 Examples • Process in ring 3 accesses data segment ‣ access bracket: (2, 4) ‣ What operations can be performed? • Process in ring 5 accesses same data segment ‣ What operations can be performed? • Process in ring 5 accesses procedure segment ‣ access bracket (2, 4) ‣ call bracket (4, 6) ‣ Can call be made? ‣ How do we determine the new ring? ‣ Can new procedure segment access the data segment above?

CSE543 - Introduction to Computer and Network Security Page 53 Multics Segments • Named segments are protected by access control lists and MLS protections ‣ Hierarchically arranged ‣ Precursor to hierarchical file systems • Memory segment access is controlled by hardware monitor ‣ Multics hardware retrieves segment descriptor word (SDW) • Like a file descriptor ‣ Based on rights in the SDW determines whether can access segment • Master mode (like root) can override protections

CSE543 - Introduction to Computer and Network Security Page 54 Multics Vulnerability Analysis • Detailed security analysis covering ‣ Hardware ‣ Software ‣ Procedural features (administration) • Good news ‣ Design for security ‣ System language prevents buffer overflows • Defined buffer sizes ‣ Hardware features prevent buffer overflows • Addressing off segment is an error • Stack grows up ‣ System is much smaller than current UNIX systems

CSE543 - Introduction to Computer and Network Security Page 55 Vulnerabilities Found

• Hardware ‣ Indirect addressing -- incomplete mediation • Check direct, but not indirect address ‣ Mistaken modification introduced the error • Software ‣ Ring protection (done in software) • Argument validation was flawed • Certain type of pointer was handled incorrectly ‣ Master mode transfer • For performance, run master mode program (signaler) in user ring • Development assumed trusted input to signaler -- bad combo • Procedural ‣ door insertion goes undetected

CSE543 - Introduction to Computer and Network Security Page 56 Final Project • Due: December 19th, 5:00pm (5:01pm gets 0 points) • Email to [email protected] • Project must include ‣ 5-10 page write-up of publishable quality and content. ‣ Effort should match number of people in group. ‣ CC all members (I will email back your final grade) ‣ Source code/data should be tarred and put on web for downloading. ‣ No presentations!

• 10% extra credit for turning it in at the final exam!!!!

CSE543 - Introduction to Computer and Network Security Page 57 Final • Time: either this weekend (you name a time), 2 hours or Tuesday, December 16 - 4:40pm-6:30pm (230 ARTS) ‣ Note: everyone must agree to change. • Content: all content in the class, more focus on topics since the midterm (100 total pts) ‣ 14 short answer (3 pts) = 42pts ‣ 4 long answer (7 pts) = 28pts ‣ 3 word problems (10 pts) = 30pts

• Remainder of today: questions?

CSE543 - Introduction to Computer and Network Security Page 58