In development
Example task
Contract review
Conceptual example
Receive the contract
A draft agreement arrives for review
Mark up the draft
Flag risky clauses and propose edits
Score the redline
Compare against expected issues
What the task measures.
Score the finished work product against the issues a reviewing lawyer expects to see. Record each document the model read and each edit it made.
A draft can read well while missing a clause that matters. Scoring the finished redline against expected issues tests whether the model caught what a lawyer would.
Work with Kimpton.
Engagements are custom. We agree on the task, inputs, outcomes, and rewards before we build the environment.