Choose an evaluation contract that matches the decision.
Darwin can recruit and calibrate evaluators, preserve blinded overlap, adjudicate disagreements, and settle accepted work. The template determines the unit of work, evidence standard, cadence, and acceptance boundary.
One-time evaluation
Use a bounded evaluation when you need a decision about a specific model, dataset, release, or risk surface.
Response ranking
Run a one-time blinded AI model response ranking against this rubric. Recruit calibrated evaluators, require overlap, capture scores and rationale, adjudicate disagreements, and report agreement quality.
Evaluation rubric.pdfStart this goalOne-timeBenchmark build
Build a one-time expert-authored AI benchmark with difficult tasks, reference answers, provenance, contamination checks, independent review, versioning, and rights to accepted items.
Start this goalOne-timeTrajectory review
Run a one-time review of AI agent trajectories. Grade outcome quality, tool use, efficiency, policy compliance, and recovery, with reproducible evidence for every material failure.
Start this goalOne-timeRed-team engagement
Run a one-time AI red-team engagement. Define the allowed scope, recruit qualified testers, require reproducible non-duplicate findings, calibrate severity, protect disclosure, and retest accepted fixes.
Start this goalContinuous evaluation
Use recurring coverage when releases and production behavior keep changing. Each cycle is versioned so quality trends and regressions remain comparable.
Weekly regression evaluation
Run weekly AI model regression evaluations for ten weeks. Use the same versioned suite and thresholds, review material failures, compare deltas by slice, and return a weekly release recommendation.
Start this goalMonthlyMonthly quality audit
Run a monthly AI model quality audit for six months. Sample production behavior, cover priority slices, track drift and emerging failures, document evidence, and assign prioritized follow-up actions.
Start this goalCustom
Use a custom evaluation when the unit of work, evidence standard, or decision process does not fit a repeatable template.