pytest Documentation Release 4.6 holger krekel, trainer and consultant, http://merlinux.eu Nov 25, 2020 Contents 1 Installation and Getting Started3 1.1 Install pytest ..............................................3 1.2 Create your first test...........................................3 1.3 Run multiple tests............................................4 1.4 Assert that a certain exception is raised.................................4 1.5 Group multiple tests in a class......................................5 1.6 Request a unique temporary directory for functional tests........................5 1.7 Continue reading.............................................6 2 Usage and Invocations 7 2.1 Calling pytest through python -m pytest .............................7 2.2 Possible exit codes............................................7 2.3 Getting help on version, option names, environment variables.....................7 2.4 Stopping after the first (or N) failures..................................8 2.5 Specifying tests / selecting tests.....................................8 2.6 Modifying Python traceback printing..................................9 2.7 Detailed summary report.........................................9 2.8 Dropping to PDB (Python Debugger) on failures............................ 12 2.9 Dropping to PDB (Python Debugger) at the start of a test........................ 12 2.10 Setting breakpoints............................................ 12 2.11 Using the builtin breakpoint function.................................. 13 2.12 Profiling test execution duration..................................... 13 2.13 Creating JUnitXML format files..................................... 13 2.14 Creating resultlog format files...................................... 16 2.15 Sending test report to online pastebin service.............................. 16 2.16 Early loading plugins........................................... 17 2.17 Disabling plugins............................................. 17 2.18 Calling pytest from Python code..................................... 17 3 Using pytest with an existing test suite 19 3.1 Running an existing test suite with pytest................................ 19 4 The writing and reporting of assertions in tests 21 4.1 Asserting with the assert statement.................................. 21 4.2 Assertions about expected exceptions.................................. 22 4.3 Assertions about expected warnings................................... 23 4.4 Making use of context-sensitive comparisons.............................. 23 i 4.5 Defining your own explanation for failed assertions........................... 24 4.6 Assertion introspection details...................................... 25 5 pytest fixtures: explicit, modular, scalable 27 5.1 Fixtures as Function arguments..................................... 27 5.2 Fixtures: a prime example of dependency injection........................... 28 5.3 conftest.py: sharing fixture functions............................... 29 5.4 Sharing test data............................................. 29 5.5 Scope: sharing a fixture instance across tests in a class, module or session............... 29 5.6 Higher-scoped fixtures are instantiated first............................... 31 5.7 Fixture finalization / executing teardown code.............................. 32 5.8 Fixtures can introspect the requesting test context............................ 33 5.9 Factories as fixtures........................................... 34 5.10 Parametrizing fixtures.......................................... 35 5.11 Using marks with parametrized fixtures................................. 38 5.12 Modularity: using fixtures from a fixture function............................ 38 5.13 Automatic grouping of tests by fixture instances............................ 39 5.14 Using fixtures from classes, modules or projects............................ 41 5.15 Autouse fixtures (xUnit setup on steroids)................................ 42 5.16 Overriding fixtures on various levels................................... 43 6 Marking test functions with attributes 47 6.1 Registering marks............................................ 47 6.2 Raising errors on unknown marks.................................... 48 7 Monkeypatching/mocking modules and environments 49 7.1 Simple example: monkeypatching functions.............................. 49 7.2 Global patch example: preventing “requests” from remote operations................. 49 7.3 Monkeypatching environment variables................................. 50 7.4 API Reference.............................................. 51 8 Temporary directories and files 53 8.1 The tmp_path fixture......................................... 53 8.2 The tmp_path_factory fixture................................... 54 8.3 The ‘tmpdir’ fixture........................................... 54 8.4 The ‘tmpdir_factory’ fixture....................................... 55 8.5 The default base temporary directory.................................. 55 9 Capturing of the stdout/stderr output 57 9.1 Default stdout/stderr/stdin capturing behaviour............................. 57 9.2 Setting capturing methods or disabling capturing............................ 57 9.3 Using print statements for debugging.................................. 58 9.4 Accessing captured output from a test function............................. 58 10 Warnings Capture 61 10.1 @pytest.mark.filterwarnings ................................ 62 10.2 Disabling warnings summary...................................... 63 10.3 Disabling warning capture entirely................................... 63 10.4 DeprecationWarning and PendingDeprecationWarning......................... 63 10.5 Ensuring code triggers a deprecation warning.............................. 64 10.6 Asserting warnings with the warns function............................... 64 10.7 Recording warnings........................................... 65 10.8 Custom failure messages......................................... 66 10.9 Internal pytest warnings......................................... 66 ii 11 Doctest integration for modules and test files 69 11.1 Encoding................................................. 70 11.2 Using ‘doctest’ options.......................................... 70 11.3 Output format.............................................. 71 11.4 pytest-specific features.......................................... 71 12 Skip and xfail: dealing with tests that cannot succeed 73 12.1 Skipping test functions.......................................... 73 12.2 XFail: mark test functions as expected to fail.............................. 76 12.3 Skip/xfail with parametrize....................................... 78 13 Parametrizing fixtures and test functions 81 13.1 @pytest.mark.parametrize: parametrizing test functions................... 81 13.2 Basic pytest_generate_tests example............................. 83 13.3 More examples.............................................. 84 14 Cache: working with cross-testrun state 85 14.1 Usage................................................... 85 14.2 Rerunning only failures or failures first................................. 85 14.3 Behavior when no tests failed in the last run............................... 87 14.4 The new config.cache object....................................... 88 14.5 Inspecting Cache content......................................... 89 14.6 Clearing Cache content.......................................... 90 14.7 Stepwise................................................. 90 15 unittest.TestCase Support 91 15.1 Benefits out of the box.......................................... 91 15.2 pytest features in unittest.TestCase subclasses......................... 92 15.3 Mixing pytest fixtures into unittest.TestCase subclasses using marks............. 92 15.4 Using autouse fixtures and accessing other fixtures........................... 93 16 Running tests written for nose 97 16.1 Usage................................................... 97 16.2 Supported nose Idioms.......................................... 97 16.3 Unsupported idioms / known issues................................... 97 17 classic xunit-style setup 99 17.1 Module level setup/teardown....................................... 99 17.2 Class level setup/teardown........................................ 99 17.3 Method and function level setup/teardown................................ 100 18 Installing and Using plugins 101 18.1 Requiring/Loading plugins in a test module or conftest file....................... 102 18.2 Finding out which plugins are active................................... 102 18.3 Deactivating / unregistering a plugin by name.............................. 102 19 Writing plugins 103 19.1 Plugin discovery order at tool startup.................................. 103 19.2 conftest.py: local per-directory plugins................................. 104 19.3 Writing your own plugin......................................... 104 19.4 Making your plugin installable by others................................ 105 19.5 Assertion Rewriting........................................... 105 19.6 Requiring/Loading plugins in a test module or conftest file....................... 106 19.7 Accessing another plugin by name.................................... 107 19.8 Registering custom markers....................................... 107 iii 19.9 Testing plugins.............................................. 107 20 Writing hook functions 111 20.1 hook function validation and execution................................. 111 20.2 firstresult: