Commit graph

3 commits

Author SHA1 Message Date
darrenhinde
0d1718e551 fix(evals): use test_tmp directory for test artifacts and add cleanup
- Create evals/test_tmp/ directory for test-created files
- Update all tests to write files to test_tmp/ instead of project root
- Add cleanup logic to test runner (before and after tests)
- Add comprehensive HOW_TESTS_WORK.md documentation
- Remove src/ directory that was accidentally created by tests

This prevents tests from polluting the project directory with test artifacts.
2025-11-26 10:14:29 +00:00
darrenhinde
e43eaff317 feat(evals): add BehaviorEvaluator and improve test validation
- Add BehaviorEvaluator for validating agent behavior expectations:
  - mustUseTools/mustNotUseTools validation
  - minToolCalls/maxToolCalls validation
  - requiresApproval/requiresContext/shouldDelegate checks

- Fix test schema to allow behavior alone (without expectedViolations)
- Add 41 unit tests for BehaviorEvaluator, TestCaseSchema, and YAML loading
- Improve test-runner event logging with meaningful tool/message details
- Fix glob import for ESM compatibility
- Add new test cases for bash, context loading, and approval scenarios
2025-11-26 01:10:51 +00:00
darrenhinde
cc96acc50e feat: add 5 essential workflow tests and reorganize with agents/ structure
- Add 5 workflow tests (task, context loading, stop-on-failure, conversational)
- Add 1 shared test (agent-agnostic approval gate)
- Reorganize: opencode/ → agents/ for clearer structure
- Rename: sdk-tests/ → tests/ for simplicity
- Update test runner paths to new structure
- Add comprehensive documentation (test plan, guides, migration)
- Update all documentation with new paths

Total: 13 tests (was 6, added 7)
Coverage: +31% workflow compliance
All tests passing: 11/11 (100%)
2025-11-22 15:20:59 +00:00