EnvironmentHarness is available open source today. It runs agents in persistent environments and records what each participant observed, attempted and changed. You provide the environment rules, the agent programs and the grading method.
View EnvironmentHarness on GitHub
How an environment works
An environment implements four methods. initialize builds the starting state. observe returns what one participant can see. resolve applies the submitted actions and commits a transition with its outcomes and rewards. intervene applies a declared change to the state. Work against an external system, such as a simulator, a game or a market, runs through an operation that executes once and returns a receipt.
What gets recorded
Each participant receives its own observation of the shared state. Every action is recorded as attempted, then as executed. An action that fails validation, arrives after its deadline or acts on a stale observation is blocked, and the record keeps the reason. Each event is appended to a journal in which every entry carries the hash of the entry before it, so a record that was altered or removed no longer verifies.
Checkpoints, branches and experiments
A session can be checkpointed and branched with a declared intervention. The branch keeps its lineage, so two futures from the same state can be compared turn by turn. An experiment expands scenarios and trials into concurrent sessions with deterministic seeds under one frozen configuration.
Trajectories and datasets
A session's journal projects into a portable trajectory. Trajectories freeze into snapshots, and snapshots are selected into datasets identified by a content digest. A training dataset admits only complete runs recorded for training, so evaluation and held-out runs cannot be relabeled as training data. The harness exports the dataset, and your trainer consumes it.
Get started
The quickstart builds a synthetic evidence store and opens it in a local viewer. It needs no account, model API key or paid service. Its values demonstrate the data model and do not measure model quality.
- python -m pip install "environment-harness[server]"
- environment-harness --store ./environment-sessions quickstart --turns 3
- environment-harness --store ./environment-sessions serve --open