Generative Models

These models are designed to create new, original content. We discuss various types of generative models below. 

Autoregressive Models

Generate data step-by-step, predicting the next element from previously generated elements. An autoregressive model is a type of generative model that predicts the next element in a sequence based on all of the elements that came before it. The word “autoregressive” comes from “auto” (self) and “regressive” (regression, or prediction from past values).

  • Use cases: Language modeling, music composition, sequential data generation.

Large Language Model (LLM)

LLMs are a specific, powerful class of autoregressive model. The key difference isn’t in the method of generation but in the scale and architecture of the model. “autoregressive” is the method of generation, and “LLM” is the type of model that uses that method at an unprecedented scale. All LLMs are autoregressive, there are many autoregressive models that are not large enough to be considered LLMs. Eg. the ones used for time-series forecasting in economics or older, smaller language models. 

Diffusion Models

They learn to reverse a gradual noise-adding process, starting from pure noise to create realistic samples.

  • Examples: DALL·E 2, Stable Diffusion.
  • Use cases: High-quality image generation, video synthesis.
Flow-Based Models

Use invertible transformations to map between data and latent space with exact likelihood estimation.

  • Examples: RealNVP, Glow.
  • Use cases: Image generation with exact log-likelihood, scientific simulations.
Generative Adversarial Networks (GANs)

Uses two competing neural networks—a generator and a discriminator—to create highly realistic new data. Generator network creates new data and a discriminator network tries to tell if the data is real or fake. This adversarial process drives the generator to produce highly authentic content, such as images.

Use cases: Image synthesis (e.g., faces, art), video generation, super-resolution.

Variational Autoencoders (VAEs):

After learning a latent space compressed representation of data using probabilistic encoding and decoding they generate new samples by sampling from this learned latent space. This is a probabilistic approach that is effective for tasks like generating new variations of existing data.

  • Use cases: Image generation, text generation, anomaly detection.