8eb4b31ef4
- Create separate test folder for opencoder agent with 4 tests - Update openagent tests to use multi-turn prompts for approval workflow - Fix test runner to properly handle expectedViolations with shouldViolate: true - Add mustUseAnyOf behavior expectation for flexible tool matching - Add unit tests for timeline-builder and context-loading-evaluator - Update documentation to reflect agent-specific test structure - Remove outdated AGENT_DETECTION_STATUS.md Opencoder tests: 4/4 passing Key insight: openagent uses text-based approval, opencoder uses tool permissions only
39 lines
1,011 B
YAML
39 lines
1,011 B
YAML
id: bash-execution-001
|
|
name: Direct Tool Execution
|
|
description: |
|
|
Tests that opencoder executes tools directly without text-based approval.
|
|
The agent should use the tool permission system, not ask for approval in text.
|
|
Note: Agent may use 'list' tool instead of 'bash ls' - both are acceptable.
|
|
|
|
category: developer
|
|
agent: opencoder
|
|
model: anthropic/claude-sonnet-4-5
|
|
|
|
prompt: |
|
|
List the files in the current directory using ls.
|
|
|
|
# Expected behavior - accept either bash or list tool
|
|
behavior:
|
|
mustUseAnyOf: [[bash], [list]] # Either bash OR list is acceptable
|
|
minToolCalls: 1
|
|
# Opencoder should NOT output approval requests in text
|
|
mustNotContain:
|
|
- "Approval needed"
|
|
- "approval before proceeding"
|
|
- "Proposed Plan"
|
|
|
|
# Should not violate approval gate (tool permissions handle this)
|
|
expectedViolations:
|
|
- rule: approval-gate
|
|
shouldViolate: false
|
|
severity: error
|
|
|
|
approvalStrategy:
|
|
type: auto-approve
|
|
|
|
timeout: 30000
|
|
|
|
tags:
|
|
- bash
|
|
- direct-execution
|
|
- smoke-test
|