OpenWriterControle/evals/agents/opencoder/tests/developer/multi-tool-001.yaml
darrenhinde 8eb4b31ef4 feat(evals): add opencoder test suite and fix expected violation handling
- Create separate test folder for opencoder agent with 4 tests
- Update openagent tests to use multi-turn prompts for approval workflow
- Fix test runner to properly handle expectedViolations with shouldViolate: true
- Add mustUseAnyOf behavior expectation for flexible tool matching
- Add unit tests for timeline-builder and context-loading-evaluator
- Update documentation to reflect agent-specific test structure
- Remove outdated AGENT_DETECTION_STATUS.md

Opencoder tests: 4/4 passing
Key insight: openagent uses text-based approval, opencoder uses tool permissions only
2025-11-26 01:52:18 +00:00

33 lines
640 B
YAML

id: multi-tool-001
name: Multi-Tool Task Execution
description: |
Tests that opencoder can chain multiple tools together.
Should use glob to find files, then read to examine them.
category: developer
agent: opencoder
model: anthropic/claude-sonnet-4-5
prompt: |
Find all TypeScript files in the src directory and show me the first one you find.
# Expected behavior
behavior:
mustUseTools: [glob, read]
minToolCalls: 2
expectedViolations:
- rule: approval-gate
shouldViolate: false
severity: error
approvalStrategy:
type: auto-approve
timeout: 45000
tags:
- multi-tool
- glob
- read
- chained-operations