Skip to main content

Categorical Data and Binning

A categorical dtype represents a finite vocabulary. It can reduce memory, make allowed values explicit, and preserve a meaningful order.

from pandas.api.types import CategoricalDtype

priority_type = CategoricalDtype(
    categories=["low", "medium", "high"],
    ordered=True,
)
tasks["priority"] = tasks["priority"].astype(priority_type)

Ordered categoricals support ordering and range comparisons according to the declared category sequence. Unordered categoricals represent membership without claiming that one category is greater than another.

Validate the vocabulary​

Values outside the declared categories become missing during conversion. Check for unknown values before or immediately after casting, and distinguish an unknown category from a genuinely missing observation.

Bin continuous values​

ages["band"] = pd.cut(
    ages["age"],
    bins=[0, 18, 35, 65, float("inf")],
    labels=["child", "young_adult", "adult", "senior"],
    right=False,
)

ages["quartile"] = pd.qcut(ages["age"], q=4, duplicates="drop")
  • cut uses value boundaries chosen from domain meaning.
  • qcut uses sample quantiles to target similarly populated bins.

Record interval closure and edge policy. Binning loses information and can make small input changes look discontinuous; keep the original numeric value.

Encode for models​

get_dummies creates indicator columns, but encoding belongs inside the model's training and inference pipeline when category vocabularies must remain identical. Do not infer production feature columns independently from each batch.

Source​