darrenhinde
|
e43eaff317
|
feat(evals): add BehaviorEvaluator and improve test validation
- Add BehaviorEvaluator for validating agent behavior expectations:
- mustUseTools/mustNotUseTools validation
- minToolCalls/maxToolCalls validation
- requiresApproval/requiresContext/shouldDelegate checks
- Fix test schema to allow behavior alone (without expectedViolations)
- Add 41 unit tests for BehaviorEvaluator, TestCaseSchema, and YAML loading
- Improve test-runner event logging with meaningful tool/message details
- Fix glob import for ESM compatibility
- Add new test cases for bash, context loading, and approval scenarios
|
2025-11-26 01:10:51 +00:00 |
|
darrenhinde
|
cc96acc50e
|
feat: add 5 essential workflow tests and reorganize with agents/ structure
- Add 5 workflow tests (task, context loading, stop-on-failure, conversational)
- Add 1 shared test (agent-agnostic approval gate)
- Reorganize: opencode/ → agents/ for clearer structure
- Rename: sdk-tests/ → tests/ for simplicity
- Update test runner paths to new structure
- Add comprehensive documentation (test plan, guides, migration)
- Update all documentation with new paths
Total: 13 tests (was 6, added 7)
Coverage: +31% workflow compliance
All tests passing: 11/11 (100%)
|
2025-11-22 15:20:59 +00:00 |
|