D1750R0 A Proposal to Add Process Management to the C++ Standard Library Klemens Morgenstern, Jeff Garland, Elias Kosunen, Fatih Bakir June 17, 2019 Contents 1 Revision History 2 2 Introduction 2 3 Motivation and Scope 2 4 Domain Terminology 3 4.1 Processes . 3 4.2 Process Groups . 3 4.3 Pipes . 3 4.4 Environment and Command Line Arguments . 3 5 Survey of facilities in other standard libraries 3 5.1 C/C++ system function . 3 5.2 Java................................................... 3 5.3 Python . 4 5.4 Rust . 4 5.5 Node.js . 4 6 Design Discussion & Examples 5 6.1 Concept ProcessLauncher ...................................... 5 6.2 Concept ProcessInitializer .................................... 5 6.3 Class process .............................................. 6 6.4 Class process_group ......................................... 7 6.5 Class process_io ........................................... 8 6.6 Class environment ........................................... 9 7 Design Decisions 10 7.1 Namespace std versus std::process . 10 7.2 Using a builder method to create . 11 7.3 wait or join .............................................. 11 7.4 Native Operating System Handle Support . 11 7.5 Pipe close and EOF . 11 7.6 Security and User Management Implications . 11 7.7 Extensibility . 12 7.8 Error Handling . 12 7.9 Synchronous Versus Asynchronous and Networking TS . 12 7.10 Integration of iostream and pipes . 12 8 Technical Specification 12 8.1 Header <process> Synopsis . 12 8.2 Class process .............................................. 13 8.3 Class process_error ......................................... 15 8.4 Class process_group ......................................... 15 8.5 Class process_io ........................................... 17 8.6 Class environment ........................................... 17 8.7 Class process_limit_handles .................................... 18 8.8 Namespace this_process ....................................... 19 8.9 Header <pstream> Synopsis . 20 8.10 Classes basic_pipe_read_end, basic_pipe_write_end, basic_pipe . 23 8.11 Class templates basic_pipebuf, basic_opstream, basic_ipstream and basic_pstream . 25 8.12 Classes async_pipe_read_end, async_pipe_write_end, async_pipe . 29 9 Open Questions 33 9.1 Core language impact . 33 10 Acknowledgements 33 11 References 33 1 1 Revision History Initial release. 2 Introduction The creation and management of processes is a widely used and fundamental tool in computing systems. Unfortunately, C++ does not have a portable way to create and manage processes. Most other language standard libraries support facilities to wrap the complexities in support of application programmers. The functionality has been on standard library wishlists going back to 2007. The proposal is based on boost.process which provides cross-platform implementation experience. 3 Motivation and Scope We propose a library to facilitate the following functionality: • create child processes on the current machine • setup streams for communication with child stdout and stderr • communicate with child processes through streams • wait for processes to exit • terminate processes • capture the return result of a child process • manage groups of processes • optionally associate the child-process to the parent-child to children die with their parents, but not vice- versa. The following illustrates an example usage of the proposed library. #include <process> int main() { std::vector<std::string> args { "--version", "--std=c++2a" }; try { std::ipstream pipe_stream; //capture stdout, leave the others as the OS defaults std::process child("gcc", args, std::process_io({}, pipe_stream, {})); std::string line; while (pipe_stream && std::getline(pipe_stream, line) && !line.empty()) { std::cerr << line << std::endl; } child.wait(); } catch (const std::process_error& e) { std::cerr << e.what(); } } Additional user examples can be seen online. 2 4 Domain Terminology 4.1 Processes A process is an instance of a program in execution. A process has at least one thread. A process starts execution in the thread that invokes its main function. A child process is the result of another process creating or spawning the child. 4.2 Process Groups Process groups allow for managing a set of processes at the operating system level. This allows behavior such as process termination to be automatically coordinated by the operating system. For example, child processes in a group can be set to terminate together. 4.3 Pipes A pipe is a unidirectional, serial communication line across processes. A pipe has two ends: a write end and a read end. A pipe is buffered. The size of the buffer is implementation defined. When there’s no data in the buffer, the pipe is called empty. When the buffer is full, the pipe is called full. Reading from an empty pipe is a blocking operation. Writing to a pipe resumes any blocked threads that are waiting to read on that pipe. Writing to a full pipe is a blocking operation. Reading from a pipe resumes any blocked threads that are writing to that pipe. If there are multiple threads reading or writing from the same pipe at the same time the order in which they read the data is unspecified. 4.4 Environment and Command Line Arguments Creation of a child process sometimes involves modifying the environment for the child process. This proposal references a current proposal for referencing a process environment. However, the proposal P1275 would need to be enhanced to support multiple instances of environments for access and modification of child process environment. This proposal currently contains a synopsis of similar functionality. 5 Survey of facilities in other standard libraries 5.1 C/C++ system function C and C++ currently provide a minimal process launching capability via the system function. The C++ function takes a const char* parameter that represents the command string to execute and an integer return code that signifies the execution return status. int result = system("echo \"foo\" > bar.txt"); if (result == 0) { //success } This minimal facility lacks many aspects of process control needed for even basic applications, including access to the standard streams (stdin, stdout, stderr) of the child. In addition it uses the system shell to interpret the command, which is a huge security hazard because of shell injection. 5.2 Java Java provides a ProcessBuilder and stream piping facilities similar to what is proposed here. // ProcessBuilder takes variadic string arguments 3 // or a List<String> var builder = new ProcessBuilder("/bin/cat", "-"); // start()-method will spawn the process // Standard streams are piped automatically Process p = builder.start(); // Write into process stdin new OutputStreamWriter(p.getOutputStream()) .write("foo\n") .close(); // close() needed to flush the buffer // Read from stdout var reader = new BufferedReader( new InputStreamReader(p.getInputStream())); String output = reader.readLine(); assert output == "foo"; System.out.println("Exited with " + p.exitValue()) 5.3 Python from subprocess import run # Command line arguments are all passed in a single list # Standard streams aren’t piped by default result = run([ ’/bin/cat’, ’-’ ], input=’foo\n’, capture_output=True) assert result.stdout == ’foo’ print("Exited with", result.returncode) 5.4 Rust As with other languages Rust provides the ability to pipe the results of the process into the parent. use std::process::{Command, Stdio}; let mut child = Command("/bin/cat") .arg("-") // .args() also available, taking a range // strings passed to .arg() are escaped .stdin(Stdio::piped()) .stdout(Stdio::piped()) .spawn()?; // ?-operator is for error handling child.stdin.as_mut()?.write_all(b"foo\n")?; // .wait_with_output() will, well, wait // child.stdout/stderr exposes standard streams directly let output = child.wait_with_output()?; assert_eq!(b"foo", output.stdout.as_slice()); println!("Exited with {}", output.status.code.unwrap()); 5.5 Node.js const { spawn } = require(’child_process’); 4 // First argument is argv[0], rest of argv passed in a list const p = spawn(’/bin/cat’, [’-’]); p.stdin.write(’foo\n’); // Idiomatic node.js uses callbacks everywhere p.stdout.on(’data’, (data) => { assert.StrictEqual(data, ’foo\n’); }); p.on(’close’, (code) => { console.log(‘Exited with ${code}‘); }); 6 Design Discussion & Examples 6.1 Concept ProcessLauncher The process launcher is a class that impements the actual launch of a process. In most cases there are different versions to do this. On Linux for example, vfork can be required as an alternative for fork on low-memory devices. And while POSIX can change a user by utilizing setuid in a ProcessInitializer, Windows requires the invocation of a different function (CreateProcessAsUserA). As an example for Linux: #include <gnu_cxx_process> __gnu_cxx::vfork_launcher launcher; std::process my_proc("/bin/programm", {}, launcher); or for Windows: __msvc::as_user_launcher{"1234-is-not-a-safe-user-token"}; std::process my_proc("C:\program", {}, launcher); In addition libraries may provide their launchers. The requirement is that there is an actual process with a pid as the result of launching the process. Furthermore, the fact that the launcher has a well-specified launch function allows to launch a process like this: std::process_launcher launcher; auto proc = launcher.launch("/bin/foo", {}); Both versions make sense in their own way: on the one hand using the process constructor fits well in with the STL and it’s RAII classes like thread. On the other hand it actually uses a factory-class, which can be