Generative Models at a Glance

Predict the next piece, or remove noise

Generative models learn to produce new data. Autoregressive models (GPT-style transformers) generate one token at a time by predicting the next token, repeated. Diffusion models learn to remove noise step by step and generate images, audio and video by starting from pure noise. Variational autoencoders and GANs are earlier families still used in places. All are trained with the same basics in this course (losses, backpropagation, optimisers) at enormous scale, and their outputs need evaluation for quality, safety and factual accuracy.

Generative families compared

One-line summaries.

family            generates by                          typical use
autoregressive    predicting the next token repeatedly   text, code, chat
diffusion         denoising random noise step by step    images, audio, video
VAE               decoding samples from a latent space   compression, latent models
GAN               generator vs discriminator game        image synthesis (older)

Use APIs or pretrained checkpoints

Training generative models from scratch is expensive; most teams adapt pretrained models or call hosted ones.

Quick check: How do autoregressive language models generate text?

• By sorting the vocabulary
• By denoising an image
• By clustering documents
• By predicting the next token repeatedly