पाठ 11 / 30
Replication, Indexes and Headroom
यह पाठ अभी हिंदी में अनुवादित नहीं है, इसलिए अंग्रेज़ी में दिखाया जा रहा है।
Turn logical data size into provisioned disk by adding replicas, indexes, compaction space and free headroom.
लेखक: Rahul Agarwal · 15 साल से वेब डेवलपर
Logical size is not disk size
The bytes your application writes are the logical size. The disk you must provision is larger for several reasons. Replication: databases and distributed stores commonly keep multiple copies for durability and availability, often three, which triples raw usage (erasure coding in object stores reduces this overhead, but you rarely control it directly). Indexes: each secondary index stores keys plus pointers and can add a large fraction of the table size. Storage engine overhead: page fill factors, write-ahead logs, and LSM-tree compaction can temporarily need significant extra free space. Backups and snapshots consume their own storage. Headroom: running disks nearly full harms performance and leaves no room for spikes, so plan to stay well below capacity, for example 60 to 70% full. Multiply these together and the provisioned figure can easily be several times the logical size.
From logical to provisioned
Each factor is an explicit assumption you can tweak.
logical_tb = 20
replication = 3
index_overhead = 0.3 # +30% for secondary indexes
engine_overhead = 0.2 # WAL, fragmentation, compaction space
max_fill = 0.7 # keep disks at most 70% full
raw = logical_tb * (1 + index_overhead) * (1 + engine_overhead) * replication
provisioned = raw / max_fill
print(f"raw with replicas : {raw:.1f} TB")
print(f"provision at 70% : {provisioned:.1f} TB")
print(f"multiplier : {provisioned / logical_tb:.1f}x logical")
Output:
raw with replicas : 93.6 TB provision at 70% : 133.7 TB multiplier : 6.7x logical
Packing for a trip
Your clothes fit in a small bag, but you carry a bigger one: a copy of documents, space for things you will buy, and room so the zip still closes. Disks need the same spare room.
त्वरित जाँच: Logical data is 10 TB, replication factor is 3, and you want disks at most 75% full. Ignoring indexes, how much disk should you provision?
- About 13 TB
- About 30 TB
- About 40 TB
- About 22.5 TB
Answer
About 40 TB — 10 TB x 3 replicas = 30 TB raw. Dividing by 0.75 for headroom gives 40 TB.