Memory optimization
Shrinking dtypes and loading only needed columns can cut a DataFrame's memory use dramatically.
In this page:
Syntax
df.memory_usage(deep=True)
df["column"] = pd.to_numeric(df["column"], downcast="integer")Memory optimization
memory_usage(deep=True) shows each column's bytes. Downcast numbers with pd.to_numeric(downcast="integer"), convert repetitive text to category and read only usecols. These steps often reduce memory by more than half.
Note:
df.info(memory_usage="deep") prints the total.
Example: Memory optimization
import pandas as pd
df = pd.DataFrame({"id": range(1000), "grade": ["A", "B"] * 500})
before = df.memory_usage(deep=True).sum()
df["id"] = pd.to_numeric(df["id"], downcast="integer")
df["grade"] = df["grade"].astype("category")
after = df.memory_usage(deep=True).sum()
print(df.dtypes.to_dict())
print("smaller:", after < before)
# Output:
# {'id': dtype('int16'), 'grade': CategoricalDtype(categories=['A', 'B'], ordered=False)}
# smaller: TrueRelated Topics
Common Mistakes
- Ignoring deep=True for text columns
- Downcasting values that later overflow
- Loading columns you never use
Chapter Summary
- memory_usage(deep=True) measures
- Downcast numbers
- Use category for repeats
- Load only needed columns
🔒
Chapter Quiz — Complete all 6 topics to unlock
0/6 topics done
Complete these topics first: