Non-Maximum Suppression

Keep the best, drop overlapping duplicates

Detectors usually produce several overlapping boxes for the same object. Non-maximum suppression (NMS) sorts boxes by score, keeps the highest, removes others that overlap it above an IoU threshold, and repeats. A low threshold removes more boxes (risking merging nearby objects); a high threshold keeps more (risking duplicates). NMS is applied per class. Some newer detectors are trained to avoid duplicates and need little or no NMS.

NMS on five boxes covering two objects, run

I ran this on CPU with Python 3, OpenCV 5.0.0, scikit-image 0.26.0, PyTorch 2.14.1 and torchvision 0.29.1, using scikit-image's bundled sample photos and torchvision's published pretrained weights. Three boxes cover the first object and two cover the second. torchvision's NMS at IoU threshold 0.5 keeps box 0 (score 0.92) and box 3 (score 0.81), one per object.

import torch
from torchvision.ops import nms
boxes = torch.tensor([[50, 50, 150, 150], [55, 52, 152, 148], [60, 58, 158, 160],   # three boxes on object 1
                      [300, 80, 380, 200], [305, 85, 382, 198]], dtype=torch.float)  # two on object 2
scores = torch.tensor([0.92, 0.88, 0.75, 0.81, 0.79])
keep = nms(boxes, scores, iou_threshold=0.5)
print("kept box indices:", keep.tolist())
print("kept scores:", scores[keep].tolist())

Output:

kept box indices: [0, 3]
kept scores: [0.9200000166893005, 0.8100000023841858]

Tune the threshold for crowded scenes

In dense scenes (crowds, shelves) a higher NMS threshold avoids merging neighbouring objects.

Quick check: What does NMS remove?

• Lower-scoring boxes that heavily overlap a higher-scoring box
• All boxes below 0.99
• Every other box
• Ground-truth labels