| Answer / output quality | Use business-approved examples and distinguish correct, incorrect, unsupported and should-refuse outputs. | Meet the per-metric SOW threshold while disclosing sample size and known limitations. |
|---|
| Task completion | Test the workflow end to end and verify side effects such as tickets, records, approvals or notifications. | The task completes with the expected state for each agreed fixture. |
|---|
| Latency | Measure p50/p95 on critical paths under a named load profile, not from a single request. | Stay within the agreed path-specific targets and document timeout behaviour. |
|---|
| Unit economics | Convert model/API and other variable usage into THB per successful task and model expected volume. | Cost per successful task stays below the approved ceiling with budget/usage alerts. |
|---|
| Failure paths | Simulate bad input, missing data, rate limits, timeouts, tool/API failure and retrieval without evidence. | Fail safely, avoid duplicate/incorrect actions and hand off or retry according to the agreed policy. |
|---|
| Human review | Measure work requiring review, correction, rejection or escalation by risk class. | Review rate and error-after-review stay within thresholds, with an accountable queue owner. |
|---|
| Rollback | Revert the model/prompt/config/release or disable automation and return to the manual path. | The runbook works, important data is preserved and rollback authority is documented. |
|---|