World Models in AI Systems

Metrics#

• Prediction fidelity: Long-horizon latent error; reconstruction quality where applicable
• Planning performance: Task success rate, reward, constraint violations, energy/smoothness
• Sample efficiency: Performance vs data curve; benefit of imagination steps
• Uncertainty calibration: Predictive variance vs empirical error; OOD detection quality

Experimental design#

• Use held-out seeds/environments; stress tests with rare events
• Closed-loop evaluation (act with the model) in addition to open-loop prediction
• Report compute budgets and latency; include ablations (V, M, C; rollout horizon; uncertainty method)