Skip to main content

Pandas Idioms

Readable pandas code exposes the sequence of table transformations and uses the most specific operation available.

Transformation style​

result = (
    orders.loc[lambda x: x["status"].eq("paid")]
          .assign(net=lambda x: x["gross"] - x["fee"])
          .groupby("customer_id", as_index=False)
          .agg(total_net=("net", "sum"), orders=("net", "size"))
          .sort_values("total_net", ascending=False)
)

Break a chain into named stages when it becomes difficult to inspect, reuse, or debug. Method chaining is a readability tool, not a performance guarantee.

Choose the operation​

NeedPrefer
Scalar arithmetic or typed operationvectorized expression / accessor
Map each value through a lookup or functionSeries.map / DataFrame.map
Reduce values to summariesagg
Return same-shaped group/column valuestransform
Apply a function to rows or columnsapply
Insert a whole-table function into a chainpipe
def require_positive(frame, column):
    if not frame[column].ge(0).all():
        raise ValueError(f"{column} must be non-negative")
    return frame

clean = orders.pipe(require_positive, "gross")

Vectorize before apply​

# Prefer
df["ratio"] = df["numerator"].div(df["denominator"])

# Reserve for genuinely row-dependent Python logic
df["label"] = df.apply(classify_row, axis="columns")

Row-wise apply constructs a Series per row and invokes Python repeatedly. Benchmark only after choosing a correct, clear expression and testing on representative data.

Mutation rule​

Prefer expressions returning new objects and one-step .loc assignments. Avoid chained assignment and avoid relying on inplace=True as an optimization.

Source​