Measuring Time and Memory

Measure, do not guess

Complexity analysis tells you how code scales; measurement tells you how fast it actually is. Use timeit for micro-benchmarks: it runs a statement many times, disables garbage collection during timing by default and reports total time; in a terminal, python -m timeit chooses a sensible number of loops. In Jupyter, use %timeit. For whole programs, use cProfile (python -m cProfile -s cumtime app.py) to find which functions take time, and sampling profilers such as py-spy or Scalene for production-like runs. For memory, sys.getsizeof(obj) gives the size of the container object itself (not the objects it references), so a list of strings is much larger than getsizeof reports; tracemalloc tracks allocations and peak usage, and __slots__, generators, array, and NumPy reduce memory dramatically for large data. Benchmark with realistic data sizes, run several repetitions, compare relative numbers on the same machine, and remember that a faster algorithm usually beats micro-optimisation.

timeit and tracemalloc in practice

Comparing string building and measuring peak memory.

import sys
import timeit
import tracemalloc

words = [f"word{i}" for i in range(10_000)]

def concat_plus():
    s = ""
    for w in words:
        s += w + ","          # may copy the growing string repeatedly
    return s

def concat_join():
    return ",".join(words) + ","

print(timeit.timeit(concat_plus, number=100))
print(timeit.timeit(concat_join, number=100))   # join is typically much faster

print(sys.getsizeof([]))                         # size of an empty list object
print(sys.getsizeof(words))                      # the list's pointer array only
print(sum(sys.getsizeof(w) for w in words))      # the strings themselves

tracemalloc.start()
squares = [n * n for n in range(1_000_000)]      # materialises a million ints
current, peak = tracemalloc.get_traced_memory()
print(f"peak {peak / 1e6:.1f} MB")
tracemalloc.stop()

total = sum(n * n for n in range(1_000_000))     # generator: constant memory

getsizeof is shallow

sys.getsizeof(my_list) does not include the elements. To estimate the true footprint of nested data, use tracemalloc or a library such as Pympler, and test with realistic sizes.

Quick check: What does sys.getsizeof(some_list) measure?

• The total memory of the list and all elements
• The number of elements
• Only the list object itself, including its array of references, not the referenced elements
• Disk usage