Copy-on-Write in pandas 3

Every derived object behaves like a copy

pandas 3 enables copy-on-write: any DataFrame or Series derived from another (a filter, a column, a slice) behaves as an independent copy, and data is copied lazily only when one side is modified. Modifying a subset never changes the parent, and chained assignment such as df[mask]["b"] = 0 does not update df. To change the original, use a single df.loc[mask, "b"] = value.

Subsets, loc updates and columns under copy-on-write, run

I ran this with Python 3.12.3 and pandas 3.0.6. Changing the filtered subset leaves df unchanged; df.loc updates the original; changing a column taken from df does not alter df.

import pandas as pd

df = pd.DataFrame({"a": [1, 2, 3], "b": [10, 20, 30]})
subset = df[df["a"] > 1]
subset["b"] = 0                     # modifies only the subset (copy-on-write)
print(df["b"].tolist(), subset["b"].tolist())

df.loc[df["a"] > 1, "b"] = 0        # the correct way to update df
print(df["b"].tolist())

s = df["a"]
s.iloc[0] = 100                     # s is a new object; df is unchanged
print(df["a"].tolist(), s.tolist())

Output:

[10, 20, 30] [0, 0]
[10, 0, 0]
[1, 2, 3] [100, 2, 3]

Watch for old tutorials

Code written for pandas 1.x that relied on modifying views may silently stop updating data in pandas 3; use loc for assignment.

Quick check: In pandas 3, how do you set column b to 0 where a > 1 in df?

• df[df["a"] > 1]["b"] = 0
• df.loc[df["a"] > 1, "b"] = 0
• subset = df[df["a"] > 1]; subset["b"] = 0
• df.b.iloc = 0