OpenWriterControle/evals/agents/openagent/tests/developer/ctx-review-001.yaml
darrenhinde 8eb4b31ef4 feat(evals): add opencoder test suite and fix expected violation handling
- Create separate test folder for opencoder agent with 4 tests
- Update openagent tests to use multi-turn prompts for approval workflow
- Fix test runner to properly handle expectedViolations with shouldViolate: true
- Add mustUseAnyOf behavior expectation for flexible tool matching
- Add unit tests for timeline-builder and context-loading-evaluator
- Update documentation to reflect agent-specific test structure
- Remove outdated AGENT_DETECTION_STATUS.md

Opencoder tests: 4/4 passing
Key insight: openagent uses text-based approval, opencoder uses tool permissions only
2025-11-26 01:52:18 +00:00

49 lines
1.3 KiB
YAML

id: ctx-review-001
name: Review Task with Context Loading
description: |
Tests the Execute stage context loading for code review tasks.
Validates that agent loads .opencode/context/core/workflows/review.md before reviewing code.
Critical rule from openagent.md (Line 162-193):
"Review tasks → .opencode/context/core/workflows/review.md (MANDATORY)"
category: developer
agent: openagent
prompt: |
Review the code in src/utils/math.ts and provide feedback on:
- Code quality
- Best practices
- Potential improvements
# Expected behavior
behavior:
mustUseTools: [read] # Must read context + code file
requiresApproval: false # Review is read-only, no approval needed
requiresContext: true # MUST load review.md before reviewing
minToolCalls: 1 # At least: read context
# Expected violations
expectedViolations:
- rule: approval-gate
shouldViolate: false
severity: error
description: Review is read-only, no approval needed
- rule: context-loading
shouldViolate: false
severity: error
description: Must load review.md before reviewing code
# Approval strategy
approvalStrategy:
type: auto-approve
timeout: 60000
tags:
- workflow-validation
- context-loading
- review-task
- critical-rule
- v2-schema