Lesson 10 / 33
Shared Embedding Spaces
Explain what a joint image-text embedding space is and measure alignment with cosine similarity.
By Rahul Agarwal · 15 years in web development
One space for pictures and words
If an image encoder and a text encoder both output vectors of the same length, we can train them so that an image and a caption describing it land close together, while unrelated pairs land far apart. The result is a shared (joint) embedding space. Closeness is usually measured with cosine similarity: the dot product of two vectors after each is scaled to length 1, giving a value between -1 and 1. Once such a space exists, many tasks become nearest-neighbour search: find images for a text query, find captions for an image, cluster a photo collection, or detect near-duplicates. The two encoders never generate anything; they only produce vectors. This is why joint-embedding models such as CLIP are cheap to use at scale: you embed your image collection once, store the vectors in a vector index, and every query costs one text encoding plus a fast similarity search.
Two towers, one space
A dual-encoder model runs an image tower and a text tower separately, then compares their normalised outputs in a shared space.
A map of a city
Think of the shared space as a city map where every photo and every sentence gets a pin. "Chai stall at a railway station" and a photo of exactly that are pinned on the same street, while a photo of a cricket match sits across town.
Quick check: Two unit-length vectors have a cosine similarity of 1.0. What does that mean?
- They are unrelated
- They point in opposite directions
- One of them is a zero vector
- They point in exactly the same direction
Answer
They point in exactly the same direction — Cosine similarity is 1 when the angle between the vectors is zero, i.e. they point the same way.