Commit graph

2 commits

Author SHA1 Message Date
darrenhinde
0d1718e551 fix(evals): use test_tmp directory for test artifacts and add cleanup
- Create evals/test_tmp/ directory for test-created files
- Update all tests to write files to test_tmp/ instead of project root
- Add cleanup logic to test runner (before and after tests)
- Add comprehensive HOW_TESTS_WORK.md documentation
- Remove src/ directory that was accidentally created by tests

This prevents tests from polluting the project directory with test artifacts.
2025-11-26 10:14:29 +00:00
darrenhinde
e43eaff317 feat(evals): add BehaviorEvaluator and improve test validation
- Add BehaviorEvaluator for validating agent behavior expectations:
  - mustUseTools/mustNotUseTools validation
  - minToolCalls/maxToolCalls validation
  - requiresApproval/requiresContext/shouldDelegate checks

- Fix test schema to allow behavior alone (without expectedViolations)
- Add 41 unit tests for BehaviorEvaluator, TestCaseSchema, and YAML loading
- Improve test-runner event logging with meaningful tool/message details
- Fix glob import for ESM compatibility
- Add new test cases for bash, context loading, and approval scenarios
2025-11-26 01:10:51 +00:00