W&M ScholarWorks Dissertations, Theses, and Masters Projects Theses, Dissertations, & Master Projects 2003 Domain and type enforcement in Linux Serge Edward Hallyn College of William & Mary - Arts & Sciences Follow this and additional works at: https://scholarworks.wm.edu/etd Part of the Computer Sciences Commons Recommended Citation Hallyn, Serge Edward, "Domain and type enforcement in Linux" (2003). Dissertations, Theses, and Masters Projects. Paper 1539623428. https://dx.doi.org/doi:10.21220/s2-0x9t-ag80 This Dissertation is brought to you for free and open access by the Theses, Dissertations, & Master Projects at W&M ScholarWorks. It has been accepted for inclusion in Dissertations, Theses, and Masters Projects by an authorized administrator of W&M ScholarWorks. For more information, please contact [email protected]. Domain and Type Enforcement in Linux A Dissertation Presented to The Faculty of the Department of Computer Science The College of William & Mary in Virginia In Partial Fulfillment Of the Requirements for the Degree of Doctor of Philosophy by Serge Edward Hallyn 2003 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. APPROVAL SHEET This dissertation is submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy Serge E. Hally: Approved, September 2003 Phil Kearns Thesis Advisor Weizhen Mao Xiaodong Zhang Haining Wang I / * / I/ A__ / / t - / -A | /z t Jean Michigan Technological University Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. To my mother. iii Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Table of Contents Acknowledgments xi List of Figures xiv Abstract xv 1 Introduction 2 1.1 Contributions ................................................................................................................... 2 1.2 O rg a n iz a tio n ................................................................................................................... 3 2 Background 5 2.1 Security N o m en clatu re .................................................................................................. 5 2.2 Access Control Policies .................................................................................................. 6 2.2.1 Bell-La Padula (B L P) ...................................................................................... 6 2.2.2 R ings .................................................................................................................... 8 2.2.3 Clark-Wilson ....................................................................................................... 10 2.2.4 Strict Integrity ................................................................................................... 11 2.2.5 Type Enforcement ............................................................................................ 11 iv Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 2.3 Policy R e p re s e n ta tio n ................................................................................................... 13 2.3.1 Access Control M atrix ....................................................................................... 13 2.3.2 Access Control List ............................................................................................ 13 2.3.3 Capabilities .......................................................................................................... 14 2.4 Role Based Access Control ............................................................................................ 16 2.5 U N I X ................................................................................................................................. 17 2.5.1 File System .......................................................................................................... 17 2.5.2 File A ccess .......................................................................................................... 19 2.5.3 Signal Access ....................................................................................................... 20 2.5.4 Superuser .............................................................................................................. 21 2.5.5 POSIX Capabilities ......................................................................................... 21 2.5.6 Domain and Type Enforcement ..................................................................... 23 2.6 Linux File System Architecture .................................................................................. 24 2.7 Stackable File S ystem ................................................................................................... 26 2.8 N e tw o rk in g ........................................................................................................................ 28 2.9 O ther Projects ............................................................................................................ 29 2.9.1 Linux-ACL ........................................................................................................... 29 2.9.2 L ID S ........................................................................................................................ 30 2.9.3 TE and D T E ....................................................................................................... 31 2.9.4 SELinux .................................................................................................................. 31 2.9.5 HP-LX .................................................................................................................. 32 2.10 Other Work in Security Policies ................................................................................. 33 v Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 3 DTE 34 3.1 L S M ................................................................................................................................... 35 3.1.1 LSM D esign........................................................................................................ 36 3.2 DTE Design Decisions ................................................................................................. 38 3.2.1 Entry Types ..................................................................................................... 38 3.2.2 File Type Resolution ....................................................................................... 40 3.2.3 Extended Attributes ....................................................................................... 42 3.2.4 Policy Updates .................................................................................................. 44 3.2.5 Networking ........................................................................................................ 45 3.3 D ata S tru c tu re s............................................................................................................... 46 3.4 Algorithms ...................................................................................................................... 49 3.4.1 Mount ................................................................................................................... 49 3.4.2 File Type Resolution ........................................................................................ 53 3.4.3 Inode Permission .............................................................................................. 56 3.4.4 Execve ................................................................................................................... 57 3.4.5 DTE Module Init • • 60 3.5 Configuration File .................................................................................................... 61 3.6 DTE A PI ......................................................................................................................... 63 3.7 Effectiveness ................................................................................................................... 65 4 Performance 68 4.1 LMBench R e s u lts............................................................................................................ 69 4.1.1 File System and VM Performance ............................................................. 69 vi Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 4.1.1.1 Mmap ................................................................................................... 69 4.1.1.2 File Creation ..................................................................................... 70 4.1.1.3 File Deletion ..................................................................................... 70 4.1.2 Process-Related Performance ........................................................................ 71 4.1.2.1 Null C all ............................................................................................ 71 4.1.2.2 StatQ and Open()/Close() .......................................................... 72 4.1.2.3 Signals ................................................................................................ 72 4.1.2.4 Fork ....................................................................................................... 73 4.1.2.5 Fork and E xec .................................................................................