# Grouping and Aggregating — NumPy / Pandas / scikit-learn

Source: https://www.skillbyai.com/en/numpy-pandas-sklearn/g-groupby

> Split, apply, combine.

## groupby, agg and transform

`groupby` splits rows by key, applies an aggregation and combines results: `df.groupby("city")["revenue"].sum()`. **Named aggregation** (`agg(orders=("units", "size"), ...)`) produces several clearly named statistics at once. `transform` returns a result aligned to the original rows, which is ideal for shares of a group total or group-wise normalisation.

## Group, join, reshape, resample

Most analysis questions are a combination of grouping, joining, reshaping and time-based aggregation.

![Four ideas: groupby, merge, pivot, time series.](assets/figures/numpy-pandas-sklearn/section-5-map.svg) — Figure 5.1 — Groupby, merge, pivot and resample.

## Revenue by city and each order's share, run

I ran this with Python 3.12.3 and pandas 3.0.6. Totals per city, named aggregations, and transform computing each order's share of its city's revenue.

```python
import pandas as pd

df = pd.DataFrame({
    "city": ["Delhi", "Mumbai", "Pune", "Delhi", "Pune", "Mumbai"],
    "product": ["pen", "mug", "pen", "lamp", "mug", "pen"],
    "units": [10, 4, 7, 1, 3, 5],
    "revenue": [200.0, 1000.0, 140.0, 1499.0, 750.0, 100.0],
})
print(df.groupby("city")["revenue"].sum())
print(df.groupby("city").agg(
    orders=("units", "size"),
    units=("units", "sum"),
    avg_revenue=("revenue", "mean"),
).round(1))
df["city_share"] = (df["revenue"] / df.groupby("city")["revenue"].transform("sum")).round(2)
print(df[["city", "revenue", "city_share"]])
```

Output:

```
city
Delhi     1699.0
Mumbai    1100.0
Pune       890.0
Name: revenue, dtype: float64
        orders  units  avg_revenue
city                              
Delhi        2     11        849.5
Mumbai       2      9        550.0
Pune         2     10        445.0
     city  revenue  city_share
0   Delhi    200.0        0.12
1  Mumbai   1000.0        0.91
2    Pune    140.0        0.16
3   Delhi   1499.0        0.88
4    Pune    750.0        0.84
5  Mumbai    100.0        0.09
```

## Avoid apply when a built-in exists

Built-in aggregations (sum, mean, size, nunique) are much faster than apply with a Python function.

**Quiz:** Which groupby method returns results aligned to the original rows?

- [ ] agg
- [x] transform
- [ ] size
- [ ] describe

*Answer:* transform. Same length as the input.