Lesson 33 / 33
Mixed Self-Check
Test yourself across the whole course and identify which sections to revisit.
By Rahul Agarwal · 15 years in web development
A scenario to reason through
Work through this scenario before the final question. A startup wants a WhatsApp bot that lets farmers send a photo of a diseased leaf and a voice note in Marathi, and receive advice. Which modalities arrive? An image and audio. How would you represent them? Resize and normalise the photo; resample the voice note to 16 kHz for a speech model. Cascade or end-to-end? A cascade (speech recognition, then a VLM with the transcript and photo) is easier to debug and evaluate first. How would you evaluate? Collect real photos and voice notes with agronomist-verified answers, measure WER on the Marathi transcripts and accuracy on diagnoses. What can go wrong? Blurry photos causing hallucinated diagnoses, poor accuracy on regional accents, and location metadata in photos. If any step felt unclear, revisit the matching section before moving on.
Quick-fire checks
Without scrolling back, answer: How many tokens does a 224x224 image make with 16-pixel patches? (196.) What does the temperature do in CLIP's loss? (Sharpens or softens the softmax over similarities.) Which VLM training stage trains only the projector? (Alignment.) What metric suits retrieval? (Recall@k.)
Revision like a cricket net session
Net practice is not about hitting every ball you like; it is about facing the deliveries that get you out. Spend your revision on the questions you got wrong or hesitated on, not on re-reading what you already know.
Quick check: In the farmer-bot scenario, which change most directly reduces hallucinated diagnoses from blurry leaf photos?
- Raising the speech model's sample rate
- Adding more Marathi prompts to the system message
- Removing the audio input entirely
- Asking for a clearer photo when image quality is low and allowing an 'unsure' answer
Answer
Asking for a clearer photo when image quality is low and allowing an 'unsure' answer — Detecting poor inputs and permitting uncertainty prevents the model from guessing a diagnosis from pixels it cannot see clearly.