Grouping Data

groupby partitions rows by one or more keys, applies an operation to each partition, and combines the results. Decide the desired output shape first.

Choose by shape

Operation	Result
agg	one or more summary rows per group
transform	values aligned to the original rows
filter	whole groups kept or removed
apply	flexible output; use only when the other shapes do not fit

summary = (
    sales.groupby(["region", "product"], as_index=False, dropna=False)
         .agg(revenue=("amount", "sum"), orders=("order_id", "nunique"))
)

sales["region_share"] = sales["amount"].div(
    sales.groupby("region")["amount"].transform("sum")
)

Named aggregation states both the source column and the output name. Built-in group operations are generally clearer and faster than Python callbacks.

Key decisions

• dropna=True excludes missing group keys by default; choose explicitly when missing is a meaningful group.
• observed affects whether unused categories appear for categorical groupers; set it explicitly when output shape matters across versions.
• sort=False can avoid unnecessary sorting when group order is irrelevant.
• as_index=False keeps group keys as columns, often simplifying downstream use.

MultiIndex output

Grouping by several keys can produce a MultiIndex. Keep it when hierarchical selection is useful; otherwise use as_index=False or reset_index() to return to a flat table.

apply boundary

Use GroupBy.apply for genuinely group-shaped algorithms that cannot be expressed as aggregation, transformation, filtering, windowing, or a join. Do not mutate the group object inside the function, and test index behavior explicitly.

Source

• Group by: split-apply-combine