Skip to main content

Running Subprocesses Safely

Use subprocess.run for commands that start, finish, and return a bounded result:

import subprocess

completed = subprocess.run(
    ["git", "status", "--short"],
    check=True,
    capture_output=True,
    text=True,
    encoding="utf-8",
    timeout=30,
)
print(completed.stdout)

Pass the executable and each argument as separate list elements. Without a shell, spaces and shell metacharacters in an argument remain part of that argument.

Result and failure contracts​

  • check=True raises CalledProcessError for a nonzero exit status.
  • capture_output=True captures both output streams; without text mode they are bytes.
  • timeout= bounds communication after process creation. If it expires, run() terminates and waits for the child before raising TimeoutExpired, but some platform process-creation delays are not interruptible.
  • Capturing unbounded output can exhaust memory. Inherit streams, redirect to a file, or consume incrementally when volume is unknown.

Treat expected nonzero statuses explicitly instead of setting check=False and forgetting to inspect returncode.

Environment and working directory​

import os
from pathlib import Path

child_env = os.environ.copy()
child_env["APP_MODE"] = "batch"

subprocess.run(
    ["tool", "--input", "data.json"],
    cwd=Path("workspace"),
    env=child_env,
    check=True,
)

An env mapping replaces the child's environment rather than adding to it. Start from a copy only when inheritance is intended. A controlled minimal environment reduces hidden dependencies; an inherited one may carry credentials and influence executable or library lookup.

Resolve the executable deliberately for privileged or security-sensitive code. Do not assume the caller's PATH is trustworthy.

Shell boundary​

Use shell=True only when shell language features are the actual requirement, not as a shortcut for building an argument string. With a shell, quoting, expansion, pipelines, redirection, and injection become part of the contract. Never concatenate untrusted text into a shell command. Platform rules differ, especially for Windows batch files.

For long-lived or interactive children, use Popen and own the complete lifecycle: pipe drainage, shutdown, signals, timeouts, and cleanup. Use asyncio.create_subprocess_exec when asynchronous orchestration is genuinely needed. Calling wait() while a child can fill an unread pipe may deadlock; communicate() coordinates common pipe use cases.

When several children run concurrently, Concurrency and Asyncio explains task ownership, blocking calls, and cancellation.

Argument lists are not input validation​

Avoiding a shell prevents shell expansion, not interpretation by the child program. A filename beginning with - may still be parsed as an option; use -- only if the target tool documents that convention. Windows batch files can invoke a shell even with shell=False; see the subprocess security considerations.

Failure to start (for example, a missing executable) raises OSError, often FileNotFoundError, rather than CalledProcessError. run() timeout cleanup concerns the direct child, not guaranteed termination of all descendants. With Popen.communicate(timeout=...), a timeout does not kill the child automatically: kill it and call communicate() again to drain pipes and reap it.

Source​

Explore connections

Related discussion (1)

References (1)

Referenced by (1)

More on these topics (51)

Open network