Koliseum
Financial environments for training and evaluating models. Episodes replay real research work at a fixed point in time; in live arenas a model commits, the record is sealed, and reality supplies the grade.
What an episode contains
A Koliseum episode is a controlled financial-research environment in which a model completes realistic, multi-step work using only information available at a specified point in time. Each episode starts from a clean, resettable state and records every action, citation, state change, cost, and latency.
Episodes ship as a Docker image or a programmatic API compatible with existing training and evaluation harnesses.
Private task
Expert-authored financial work with a known starting condition and expected deliverable.
Frozen information state
The model can access only sources available before the episode's timestamp.
Real tools
Search, document retrieval, market data, calculation, and artifact creation through stable interfaces.
Hidden verification
Process and outcome checks grade citations, calculations, state, formatting, and prohibited future information.
Live arenas
Episodes replay work that has already happened. Arenas run forward: the model commits before the outcome exists, the record is sealed, and the result is scored when it becomes public. Nothing in an arena can be rehearsed or backdated.
EarningsBench, the public quarterly earnings benchmark, runs as a Prediction Arena.
Prediction Arenas
A model receives only the information available at a defined cutoff. Its forecast is sealed before the outcome exists and scored when the result becomes public.
Action Arenas
A model receives a bounded task and controlled access to the systems required to complete it. Success is verified from the resulting state and downstream outcome.