Memory Sizing and Performance

Plan memory before production

Streams are memory-efficient because entries are packed into listpack nodes inside a radix tree, and field names repeated across entries are stored compactly when entries share the same fields. Still, estimate: measure MEMORY USAGE on a sample stream with realistic entries, divide by the entry count, and multiply by your retention (rate × retention time) plus headroom for consumer-group PELs and other keys. Internal node sizes are controlled by stream-node-max-bytes (default 4096) and stream-node-max-entries (default 100); the defaults suit most workloads. Performance tips: use pipelining or batching for high-rate XADD; read in batches with COUNT; avoid huge XRANGE calls over millions of entries in one request; prefer approximate trimming; keep large payloads out of entries; and remember Redis executes commands on a single main thread per instance, so one expensive command delays everyone. Benchmark with redis-benchmark or your own load test on production-like hardware.

Estimating memory from a sample

Measure, then extrapolate to the planned retention.

# after loading, say, 100,000 realistic entries into a test stream:
redis-cli MEMORY USAGE events:test SAMPLES 0     # exact bytes for the key
redis-cli XLEN events:test

# bytes_per_entry = memory / entries
# planned = bytes_per_entry x events_per_second x retention_seconds x 1.3 (headroom)
# e.g. 120 B x 2,000/s x 86,400 s x 1.3 = about 27 GB  -> retention too long for one node?

redis-cli CONFIG GET stream-node-max-*

Retention is a memory decision

Each extra hour of retention costs RAM, which is expensive. Keep only what consumers need to recover, and archive older events to cheaper storage if you need long-term history.

त्वरित जाँच: Which practice helps most with sustained high-rate XADD workloads?

• Calling XRANGE on the whole stream after each write
• Using exact MAXLEN trimming on every write
• Pipelining or batching writes and reading with COUNT
• Storing multi-megabyte payloads in entries