e43eaff317
- Add BehaviorEvaluator for validating agent behavior expectations: - mustUseTools/mustNotUseTools validation - minToolCalls/maxToolCalls validation - requiresApproval/requiresContext/shouldDelegate checks - Fix test schema to allow behavior alone (without expectedViolations) - Add 41 unit tests for BehaviorEvaluator, TestCaseSchema, and YAML loading - Improve test-runner event logging with meaningful tool/message details - Fix glob import for ESM compatibility - Add new test cases for bash, context loading, and approval scenarios
28 lines
480 B
YAML
28 lines
480 B
YAML
id: simple-bash-test
|
|
name: Simple Bash Command Test
|
|
description: Test that agent can execute a simple bash command
|
|
|
|
category: developer
|
|
agent: openagent
|
|
model: anthropic/claude-sonnet-4-5
|
|
|
|
prompt: |
|
|
List the files in the current directory using ls
|
|
|
|
behavior:
|
|
mustUseTools: [bash]
|
|
minToolCalls: 1
|
|
|
|
expectedViolations:
|
|
- rule: approval-gate
|
|
shouldViolate: false
|
|
severity: error
|
|
|
|
approvalStrategy:
|
|
type: auto-approve
|
|
|
|
timeout: 30000
|
|
|
|
tags:
|
|
- simple-test
|
|
- bash
|