USING A LOADTIME METAOBJECT PROTOCOL TO ENFORCE ACCESS CONTROL POLICIES UPON USER-LEVEL COMPILED CODE ,\ DISSERTAIION SUBMITTED TO THE SCHOOL OF C();-"ll'lTI~G OF THE UNIVERSITY OF NEWCASTLE-UPON-TYNE IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY Ian S. Welch September 2004 NEWC~STL[ UNIVCRSITY LICRARY 204 OGJ32 0 © Copyright by Ian S. Welch 200-+ All Rights Reserved 11 Dedication To my partner Jim and, my family especially my parents Stan and Claire. III Acknowledgements I would like to express my sincere gratitude to "everal people \\ho ha\e contributed in various ways to the completion of this thesis. The advice and patience of my supervisor, Dr. Robert Stroud, \\ <1" crucial in the comple­ tion of this work. Also, many thank.., to the Defence and Research Agency and particularly Dr. Peter Y. Ryan for providing the financial support, advice and encouragement needed to fini..,h my work. While at Newcastle, the advice and help given to me by a large number of people, in particular Dr. G. Morgan, Mr J. Warne, Dr. A.S. McGough, Dr. B. ArieL Dr. A. Ro­ manovsky, and Dr. C. Gaciek kept me going. Since leaving Newca..,tle and coming to \\ork at Victoria University, I have been greatly helped by the contributions of my colleagues. Participation in the Circle of Guilt has been a great help in completing but abo staying sane. Special thanks also to Margi Key" for helping proofread the final thesi" and keeping me on the straight and narrow with the use of my hyphens! Finally, I would not have started or completed this thesis if it hadn't been for the en­ couragement and support of my partner, Jim Whitman. \\ Contents Dedication iii Acknowledgements i, 1 Introduction 3 1.1 The Thesis. -l 1.2 Motivation. 1.2.1 New Problem Domains . 5 1.2.2 Man-machine Scale. 8 1.2.3 Criteria for Evaluation 9 1.2.-+ Operating System Enforcement 10 1.2.5 Conventional Object-Oriented Techniques . 10 1.2.6 In-lined Reference Monitors 12 1.2.7 Metaobject Protocols 13 1.2.8 Loadtime Metaobject Protocols 14 1.3 Contributions 15 1.3.1 Design and Implementation of a Secure Loadtime Metaobject Pro- tocol ................................ IS 1.3.2 Demonstration of the Limitations of Conventional Object-Oriented Techniques . 18 1.3.3 Evaluation of Separation of Concerns 18 1.-+ Thesis 0\ enil'\\ 18 \ 2 Related Work 20 2.1 Access Control Policies. 21 2.2 Criteria for Evaluation 2-l 2.2.1 Separation of Concerns . 2-l 2.2.2 Security Engineering Principles 26 2.3 Operating System Enforcement . 27 2.3.1 Evaluation 27 2.4 Conventional Object-oriented Security 29 2.4.1 Examples 30 2.4.2 Evaluation 35 2.5 In-lined Reference Monitors 38 2.5.1 Evaluation 39 2.6 Metaobject Protocols -l-l 2.6.1 Examples -l8 2.6.2 Evaluation 51 2.7 Goals of Thesis 5-l 3 Design and Implementation of Kava 57 3.1 Language Design of Kava. 58 3.1.1 Overview 59 3.1.2 Kava Metaobject Protocol 60 3.1.3 Context Parameters . 64 3.IA Binding 66 3.1.5 Meta Parameters 67 3.1.6 Meta-Level Cooperation 69 3.1.7 A Misbehaved Applet 70 3.2 Kava Implementation 71 3.2.1 Overview 72 3.2.2 Intercepting Class Loading . N 3.2.3 Adding Meta-Level Interceptions 75 3.2A Realising MUs 7R \'\ 3.2.5 Static Analysis ..... 79 3.2.6 Instantiating Metaobjects . 80 3.2.7 Reification .. 80 3.2.8 Static Members XI 3.2.9 Implementing Complete Mediation 3.2.10 Performance ........... 89 3.2.11 Example: Redefining Field Access. 90 3.3 Evaluation against Goals . 92 3.3.1 Enforcement upon Compiled Code. LJ2 3.3.2 Control Access to Application, Library and Operating S} -,tern Re- sources ........... 3.3.3 Clean Separation of Concerns 3.3.4 Least Privilege ... 93 3.3.5 Complete Mediation l)~ 3.3.6 Economy of Mechanism l)~ 3.4 Summary . LJ5 4 Case Study One: Standalone Application 96 -+.1 Overview . 96 ~.2 Conventional Object-oriented Security 97 -+.2.1 Application Resources 97 ~.2.2 System Resources 100 ~.3 Loadtime MOP . 104 ~.3.1 Application Resources I()~ -+.3.2 System Resources .. 106 ~.~ Improvements to the Separation of Concerns. 110 ~.5 Summary . 112 5 Case Study Two: Distributed Application 113 5.1 Overview .............. 113 5.2 Conventional Object-oriented Security 115 5.2.1 O\enie\\ ....... 115 \11 5.2.2 Secure RMI .. 116 5.2.3 Security State . 116 5.2.4 Authorisation 118 5.3 Loadtime MOP 121 5.3.1 Overview 121 5.3.2 Secure RMI . 123 5.3.3 Security State . 12-l 5.3.4 Authorisation . 125 5.4 Improvements to the Separation of Concerns. 126 5.4.1 Place Enforcement within Libraries 126 5.4.2 Place Enforcement within Proxies . 127 5.4.3 Place Enforcement within a Superclass 129 5.5 Summary ....... 129 6 Inferences from Case Studies 131 6.1 MOPs Provide Better Separation of Concerns than Conventional Object- Oriented Techniques .................... 131 6.2 Constraints to Consider when Using MOPs for Enforcement 132 6.2.1 Detailed Specifications may be Necessary . 132 6.2.2 Expressiveness of Binding Influences Complexity of Use . 132 6.2.3 Granularity ofInterfaces Constrain Policies . 133 6.2.4 Security Exceptions Break Transparency ...... 134 6.2.5 Constraints upon Applying Kava to other Languages 134 6.3 Summary ........ 135 7 Conclusions and Future Work 136 7.1 Summary of Thesis . 136 7.2 Overall Contributions 140 7.3 Future Work ..... 143 7.3.1 Improve Static Analysis 1-l-l 7.3.2 Provide Least Privilege for Metaobjects l-l-l 7.3.3 Improve Metaobject Composition I .. t~ V III 7.3.4 Make Distribution First-Class . 1~5 7.3.5 Improve Economy of Mechanism 146 7.3.6 Investigate and Improve Performance 1~6 7.3.7 Provide a Policy Language for Kava 1~7 7.4 Concluding Remarks 148 Bibliography 149 IX List of Tables 2.1 Evaluation of operating system enforcement 2X 2.2 Evaluation of conventional object-oriented techniques. 36 2.3 Evaluation of IRM techniques -Hl 2.4 Classification of MOPs -+h 2.5 Evaluation of MOP techniques )2 2.6 Evaluation of security engineering techniques )) 7.1 Summary of evaluation of security engineering techniques 138 7.2 Applying Kava to Problem Domains 1-+2 x List of Figures 1.1 Layers of an IT system 9 1.2 Overview of Kava. 17 2.1 XACML authorisation. 23 2.2 Java security architecture 30 2.3 In-lined reference monitor architecture. 39 2.4 Metaobject and object . -+) 3.1 Context hierarchy ... 65 3.2 High-level object collaboration diagram showing objects and their interac- tions when a class is loaded into the Java VM .... 72 5.1 Class diagram for self-defence security architecture 117 5.2 Collaboration diagram showing steps involved in a viewer invoking a method of a remote database ............................. 120 5.3 Collaboration diagram showing the steps involved in invoke a method of a remote database under the reflective security architecture 122 7.1 Application model 140 XI Listings 3.1 Kava metaobject protocol 59 3.2 IFieldContext interface. 66 3.3 Example of overriding base-level behaviour 66 3.4 DTD for Kava binding specification 6~ 3.5 Kavakava replacement for Java. 7J 3.6 Kava's class loader. 76 3.7 Superclass containing MLI for method execution. 87 J.8 Subclass defining its own MetaObject to use for method executions. 87 3.9 A rewritten field access. 91 4.1 Defining a channel permission ()8 4.2 Grant access to join a specific channel <)<) 4.3 Explicit check for permission to create a channel 100 4.-1- Check for implication of permissions in class Write Permission. 102 4.5 Java policy limiting Lirc's access to the network 102 4.6 Check for permission to write to the network. 103 4.7 ChannelEnforcementMetaObject controlling access to IRC channels 105 4.8 Policy file for enforcing channel access control using Kava . 107 4.9 Meta-level enforcement code tracking the creation of OutputStream in- stances by sockets. .............................. 108 4.10 Meta-\eyel enforcement code checking whether Lirc has exceeded it'-> net- work quota..................... 109 4.11 Policy file for enforcing network quota using Kava I I 1 XII Abstract This thesis evaluates the use of a loadtime metaobject protocol as a practical mechanism for enforcing access control policies upon applications distributed as user-level compiled code. Enforcing access control policies upon user-level compiled code is necessary because there are many situations where users are vulnerable to security breaches because they download and run potentially untrustworthy applications provided in the form of user-level compiled code. These applications might be distributed applications so access control for both local and distributed resources is required. Examples of potentially untrustworthy ap­ plications are Browser plug-ins, software patches, new applications, or Internet computing applications such as SETI@home. Even applications from trusted sources might be mali­ cious or simply contain bugs that can be exploited by attackers so access control policies must be imposed to prevent the misuse of resources. Additionally, system administrators might wish to enforce access control policies upon these applications to ensure that users use them in accordance with local security requirements. Unfortunately, applications devel­ oped externally may not include the necessary enforcement code to allow the specification of organisation-specific access control policies.