Skip to main content

Reading and Writing Files

File I/O crosses a boundary between program state and an external filesystem. Make the data format, encoding, mode, and failure behavior explicit.

Resource lifetime​

Use a context manager so the file is closed even when an exception occurs:

from pathlib import Path

path = Path("notes.txt")
with path.open("r", encoding="utf-8") as handle:
    for line in handle:
        process(line.rstrip("\n"))

Iterating over the file streams lines. read() and readlines() load the remaining content into memory and should be reserved for bounded inputs. rstrip("\n") removes a line ending without also discarding meaningful spaces as an unrestricted strip() would.

Text and binary modes​

Text mode decodes bytes into str; binary mode returns bytes unchanged.

text = path.read_text(encoding="utf-8")
payload = Path("image.bin").read_bytes()

Specify an encoding for durable text rather than depending on a machine's locale default. Newline translation is another text-mode behavior; pass newline= when a file format requires precise control.

The main modes are:

ModeMeaningExisting file
rreadrequired
wwritetruncated
aappendpreserved
xexclusive createmust not exist

Add b for binary mode or + for combined reading and writing. Treat w as a destructive operation.

Structured formats​

Use a format-aware library rather than hand-built string splitting:

import json

with Path("settings.json").open("r", encoding="utf-8") as handle:
    settings = json.load(handle)

Use csv for delimited data and open CSV files with newline="" as its documentation recommends. Parsing does not validate business meaning; check the resulting schema, ranges, and required fields separately.

Failures and safer replacement​

Handle only errors for which the program has a recovery policy, such as FileNotFoundError, PermissionError, decoding errors, or invalid format data. Do not collapse all failures into an empty result.

For important output, write a temporary file in the same directory, flush and close it, then replace the destination. A same-filesystem rename can prevent readers from observing a partially written file, but crash durability may also require filesystem-specific synchronization. Atomic replacement does not solve concurrent-writer coordination.

Source​