Keypoints and Feature Matching

Detect, describe, match, verify

Keypoint detectors (ORB, SIFT, AKAZE) find distinctive points such as corners; descriptors summarise each point's neighbourhood so it can be recognised in another image, even after rotation, scaling or lighting changes. Matching descriptors gives candidate correspondences, many wrong. RANSAC fits a geometric model (an affine transform or homography) while ignoring outliers. This powers panorama stitching, image alignment, visual odometry and augmented reality.

Recovering a rotation with ORB and RANSAC, run

I ran this on CPU with Python 3, OpenCV 5.0.0, scikit-image 0.26.0, PyTorch 2.14.1 and torchvision 0.29.1, using scikit-image's bundled sample photos and torchvision's published pretrained weights. The camera photo is rotated 30 degrees and scaled by 0.8. ORB finds 500 keypoints in each image and 291 cross-checked matches; RANSAC keeps 269 inliers and recovers a rotation of 29.9 degrees and scale 0.80.

import cv2, numpy as np
from skimage import data
img = data.camera()
h, w = img.shape
M = cv2.getRotationMatrix2D((w / 2, h / 2), 30, 0.8)        # rotate 30 degrees, scale 0.8
moved = cv2.warpAffine(img, M, (w, h))
orb = cv2.ORB_create(nfeatures=500)
k1, d1 = orb.detectAndCompute(img, None)
k2, d2 = orb.detectAndCompute(moved, None)
matches = sorted(cv2.BFMatcher(cv2.NORM_HAMMING, crossCheck=True).match(d1, d2), key=lambda m: m.distance)
src = np.float32([k1[m.queryIdx].pt for m in matches]); dst = np.float32([k2[m.trainIdx].pt for m in matches])
est, inliers = cv2.estimateAffinePartial2D(src, dst, method=cv2.RANSAC)
angle = np.degrees(np.arctan2(est[1, 0], est[0, 0])); scale = np.hypot(est[0, 0], est[1, 0])
print("keypoints:", len(k1), "and", len(k2), "| cross-checked matches:", len(matches))
print(f"RANSAC inliers: {int(inliers.sum())} | recovered rotation {abs(angle):.1f} deg, scale {scale:.2f}")

Output:

keypoints: 500 and 500 | cross-checked matches: 291
RANSAC inliers: 269 | recovered rotation 29.9 deg, scale 0.80

Always verify geometrically

Raw descriptor matches contain many errors; never use them without a RANSAC-style consistency check.

Quick check: What does RANSAC do in feature matching?

• Fits a geometric transform while rejecting outlier matches
• Detects corners
• Blurs the image
• Classifies objects