World Models in AI Systems

1) Encoding observations (V)#

• Purpose: Compress observations into a latent that preserves task-relevant factors of variation
• Common choices: Convolutional or transformer encoders; stochastic latents for uncertainty; discrete or continuous codes
• Design decisions:
  • Stochastic vs deterministic latents (uncertainty vs simplicity)
  • Reconstruction objective (reconstruct pixels, features, or contrastive objectives)
  • Multimodal fusion (vision + state + language)
• Pitfalls: Overfitting to pixel detail; latents that ignore controllable factors; brittle encodings under domain shift

2) Learning dynamics (M)#

• Goal: Predict next latent state and auxiliary targets (reward, termination, constraints) conditioned on action
• Options: Recurrent models, latent sequence models, diffusion/transformer predictors, mixture density outputs
• Trade-offs:
  • Expressivity vs computational cost (multi-step rollouts are expensive)
  • One-step vs multi-step training (stability vs long-horizon accuracy)
  • Stochasticity modeling (aleatoric and epistemic uncertainty)
• Techniques: Scheduled sampling, latent overshooting, consistency objectives

3) Acting and planning (C)#

• Planning styles:
• Model Predictive Control (MPC): Receding-horizon sampling/optimization with periodic replanning
• Policy learning with imagination: Train a policy using imagined trajectories from the world model
• Hybrid: Policy proposals refined by short-horizon planning
• Constraints: Safety-aware planning, cost shaping, risk sensitivity