RestrictedPython Documentation Release 4.0.0.dev0 Alexander Loechel May 10, 2017 Contents 1 The Idea behind RestrictedPython1 2 Contents 3 2.1 The Idea behind RestrictedPython....................................3 2.2 Grundlagen von RestrictedPython und der Sicherheitskonzepte von Zope2..............5 2.3 Usage of RestrictedPython........................................6 2.4 API of RestrictedPython 4.0....................................... 11 2.5 RestrictedPython 3.6.x and before.................................... 12 2.6 RestrictedPython 4+........................................... 12 2.7 Concept for a upgrade to Python 3.................................... 14 2.8 Upgrade dependencies.......................................... 16 2.9 Roadmap for RestrictedPython...................................... 17 2.10 Contributing............................................... 18 2.11 Changes................................................. 19 3 Indices and tables 23 i ii CHAPTER 1 The Idea behind RestrictedPython Python is a Turing complete programming language. To offer a Python interface for users in web context is a potential security risk. Web frameworks and Content Management Systems (CMS) want to offer their users as much exten- sibility as possible through the web (TTW). This also means to have permissions to add functionality via a Python Script. There should be additional preventive measures taken to ensure integrity of the application and the server itself, ac- cording to information security best practice and unrelated to Restricted Python. RestrictedPython defines a safe subset of the Python programming language. This is a common approach for securing a programming language. The Ada Ravenscar profile is another example of such an approach. Defining a secure subset of the language involves restricting the EBNF elements and explicitly allowing or disallowing language features. Much of the power of a programming language derives from its standard and contributed libraries, so any calling of these methods must also be checked and potentially restricted. RestrictedPython generally disallows calls to any library that is not explicit whitelisted. As Python is a scripting language that is executed by an interpreter. Any Python code that should be executed have to be explicit checked before executing a generated byte code by the interpreter. Python itself offers three methods that provide such a workflow: • compile() which compiles source code to byte code • exec / exec() which executes the byte code in the interpreter • eval / eval() which executes a byte code expression Therefore RestrictedPython offers a replacement for the python builtin function compile() (Python 2: https://docs. python.org/2/library/functions.html#compile / Python 3 https://docs.python.org/3/library/functions.html#compile). This method is defined as following: compile(source, filename, mode [, flags [, dont_inherit]]) The definition of the compile() method has changed over time, but its relevant parameters source and mode still remain. There are three valid string values for mode: 1 RestrictedPython Documentation, Release 4.0.0.dev0 • 'exec' • 'eval' • 'single' For RestrictedPython this compile() method is replaced by: compile_restricted(source, filename, mode [, flags [, dont_inherit]]) The primary parameter source has to be a ASCII or unicode string (With Python 2.6 an additional option for source was added: ast.AST for Code generation). Both methods either returns compiled byte code that the inter- preter could execute or raise exceptions if the provided source code is invalid. As compile and compile_restricted just compile the provided source code to byte code it is not sufficient to sandbox the environment, as all calls to libraries are still available. The two methods / Statements: • exec / exec() • eval / eval() have two parameters: • globals • locals which are a reference to the Python builtins. By modifying and restricting the available modules, methods and constants from globals and locals we could limit the possible calls. Additionally RestrictedPython offers a way to define a policy which allows developers to protect access to attributes. This works by defining a restricted version of: • print • getattr • setattr • import Also RestrictedPython provides three predefined, limited versions of Python’s own __builtins__: • safe_builtins (by Guards.py) • limited_builtins (by Limits.py), which provides restricted sequence types • utilities_builtins (by Utilities.py), which provides access for standard modules math, random, string and for sets. Additional there exist guard functions to make attributes of Python objects immutable –> full_write_guard (write and delete protected) 2 Chapter 1. The Idea behind RestrictedPython CHAPTER 2 Contents The Idea behind RestrictedPython Python is a Turing complete programming language. To offer a Python interface for users in web context is a potential security risk. Web frameworks and Content Management Systems (CMS) want to offer their users as much exten- sibility as possible through the web (TTW). This also means to have permissions to add functionality via a Python Script. There should be additional preventive measures taken to ensure integrity of the application and the server itself, ac- cording to information security best practice and unrelated to Restricted Python. RestrictedPython defines a safe subset of the Python programming language. This is a common approach for securing a programming language. The Ada Ravenscar profile is another example of such an approach. Defining a secure subset of the language involves restricting the EBNF elements and explicitly allowing or disallowing language features. Much of the power of a programming language derives from its standard and contributed libraries, so any calling of these methods must also be checked and potentially restricted. RestrictedPython generally disallows calls to any library that is not explicit whitelisted. As Python is a scripting language that is executed by an interpreter. Any Python code that should be executed have to be explicit checked before executing a generated byte code by the interpreter. Python itself offers three methods that provide such a workflow: • compile() which compiles source code to byte code • exec / exec() which executes the byte code in the interpreter • eval / eval() which executes a byte code expression Therefore RestrictedPython offers a replacement for the python builtin function compile() (Python 2: https://docs. python.org/2/library/functions.html#compile / Python 3 https://docs.python.org/3/library/functions.html#compile). This method is defined as following: compile(source, filename, mode [, flags [, dont_inherit]]) 3 RestrictedPython Documentation, Release 4.0.0.dev0 The definition of the compile() method has changed over time, but its relevant parameters source and mode still remain. There are three valid string values for mode: • 'exec' • 'eval' • 'single' For RestrictedPython this compile() method is replaced by: compile_restricted(source, filename, mode [, flags [, dont_inherit]]) The primary parameter source has to be a ASCII or unicode string (With Python 2.6 an additional option for source was added: ast.AST for Code generation). Both methods either returns compiled byte code that the inter- preter could execute or raise exceptions if the provided source code is invalid. As compile and compile_restricted just compile the provided source code to byte code it is not sufficient to sandbox the environment, as all calls to libraries are still available. The two methods / Statements: • exec / exec() • eval / eval() have two parameters: • globals • locals which are a reference to the Python builtins. By modifying and restricting the available modules, methods and constants from globals and locals we could limit the possible calls. Additionally RestrictedPython offers a way to define a policy which allows developers to protect access to attributes. This works by defining a restricted version of: • print • getattr • setattr • import Also RestrictedPython provides three predefined, limited versions of Python’s own __builtins__: • safe_builtins (by Guards.py) • limited_builtins (by Limits.py), which provides restricted sequence types • utilities_builtins (by Utilities.py), which provides access for standard modules math, random, string and for sets. Additional there exist guard functions to make attributes of Python objects immutable –> full_write_guard (write and delete protected) 4 Chapter 2. Contents RestrictedPython Documentation, Release 4.0.0.dev0 Grundlagen von RestrictedPython und der Sicherheitskonzepte von Zope2 Motivation für RestrictedPython Python ist eine moderne und heute sehr beliebte Programmiersprache. Viele Bereiche nutzen heute Python ganz selbstverständlich. Waren am Anfang gerade Systemadministratoren die via Python-Skripte ihre Systeme pflegten, ist heute die PyData Community eine der größten Nutzergruppen. Auch wird Python gerne als Lehrsprache verwendet. Ein Nutzungsbereich von Python unterscheidet sich fundamental: Python-Web bzw. Applikations-Server die Fremd- code aufnehmen. Zope gehörte zu den ersten großen und erfolgreichen Python-Web-Projekten und hat sich mit als erster um dieses Thema gekümmert. Während in der klassischen Software-Entwicklung aber auch in der Modelierung und Analyse von Daten drei Aspekte relevant sind: • Verständlichkeit des Programms (–> Siehe PEP 20 “The Zen of Python” https://www.python.org/dev/peps/ pep-0020/) • die Effizienz der Programmiersprache und der Ausführungsumgebung • Verfügbarkeit