gh-47798: Add a subprocess.run_pipeline() API

[gpshead]

This was a feature request from 2008. I'm undecided if we should add this feature, but it proved a good exercise and had me review a lot of existing subprocess.py code and notice a variety of other lingering subprocess issues while doing so (thus my recent spate of smaller subprocess PRs merged around when I opened this).

Summary

Read the docs in the PR for details, but it basically mirrors the run() API, just with multiple commands:

>>> from subprocess import run_pipeline
>>> run_pipeline(
...     ["gh", "issue", "list", "--state", "open", "--search", "subprocess in:title",
...      "--json", "number", "--limit", "500"],
...     ["jq", "length"],
... )
97
CompletedPipeline(commands=[['gh', 'issue', 'list', '--state', 'open', '--search', 'subprocess in:title', '--json', 'number', '--limit', '500'], ['jq', 'length']], returncodes=[0, 0])

Design notes

A future Stage(cmd, **overrides) wrapper is marked in-code as a natural extension point if per-stage stderr=STDOUT, pass_fds=, env=, etc. are wanted later.

CompletedPipeline and PipelineError are siblings of CompletedProcess and CalledProcessError under SubprocessError, not subclasses. A pipeline carries N commands and N returncodes, so any single value picked for returncode/cmd to satisfy the CalledProcessError parent contract would silently mislead existing except CalledProcessError as e: handlers -- particularly for the check=True saving-throw case where multiple stages may have failed. Users who want a single catch-all for any subprocess failure can use except SubprocessError:, the same umbrella TimeoutExpired already lives under.

Open questions

This matches the signaling behavior of run() upon timeout, namely SIGKILL. But to each process. Q: Is that rude? should we start in a particular order so that their own SIGPIPE's propagate and wait a bit before we signal each? "what does a shell do" upon Ctrl-C is probably a good way to answer that. We've never allowed people to specify the specific signal used upon timeout in run(), we could... but it'd always need to be in a "use this first, wait a bit, then resort to SIGKILL" sense to be reliable so it'd probably turn into two parameters and I doubt it would see a lot of use. I consider changing the signal to be a separate followon feature request

Alternative ideas

I was pondering the use of the | pipe operator itself between objects. But this is unnatural and undesirable for Popen instances themselves as those start upon creation. Even though I could imagine allowing that to rewire their file descriptors. It'd get gross and raise questions around unclear management of the mess. You want processes started sequentially with the actual stdout->stdin chain of connections made from the start, so a run-like API makes sense to me.

This lets people avoid using a shell.

It does not offer the same flexibility a raw Popen instance does though for people who need to do their own IO multiplexing. Though given you can provide whatever file object you want for input and output, that could still be done using this by having your own threads feed or consume those instead of relying on capture_output.

What PyPI subprocess pipe options exist?

I found two PyPI packages offering something resembling assembling pipelines of subprocesses:

• subprocess_pipe - super trivial, what anyone could easily do already, not robust.
• pipesubprocess - complicated, uses threads on all platforms, Popen-like API style.

---

Written entirely between my looking at subprocess sources, and driving a remote Claude Code for the web session and telling it what to do next. With the aid of gpshead/cpython-skills.

Co-authored-by: Claude Opus 4.5

Later picked up and updated locally using Opus 4.7.

• Issue: subprocess.pipe function #47798

---

📚 Documentation preview 📚: https://cpython-previews--142080.org.readthedocs.build/