041e3238bc
* docs: fix agent name casing in documentation - Update all instances of 'openagent' to 'OpenAgent' - Update all instances of 'opencoder' to 'OpenCoder' - Fixes #115: --agent flag is case-sensitive The --agent flag requires capitalized agent names (OpenAgent, OpenCoder) but documentation showed lowercase versions, causing confusion for users. Updated README.md and all docs/ files to use correct casing. * refactor: migrate from grok-code to big-pickle model - Remove model specification from contextscout.md subagent - Update eval YAML files to use opencode/big-pickle - Update documentation to reference big-pickle as default - Fixes compatibility as grok-code-fast is no longer available Files updated: - .opencode/agent/subagents/core/contextscout.md - evals/agents/shared/tests/golden/00-smoke-test.yaml - evals/agents/shared/tests/golden/02-delegation-test.yaml - evals/README.md - evals/EVAL_FRAMEWORK_GUIDE.md
59 lines
1.5 KiB
YAML
59 lines
1.5 KiB
YAML
id: golden-00-smoke-test
|
|
name: "Golden 00: Minimal Smoke Test - Basic Response"
|
|
description: |
|
|
Absolute bare minimum smoke test - if this fails, something is fundamentally broken.
|
|
|
|
This is the simplest possible test:
|
|
- Single prompt
|
|
- Single response
|
|
- No tool calls required
|
|
- No delegation
|
|
- No file operations
|
|
|
|
Expected completion time: 8-12 seconds (mostly LLM inference)
|
|
|
|
This test validates:
|
|
- Agent can receive a prompt
|
|
- Agent can generate a response
|
|
- Test infrastructure works
|
|
|
|
category: developer
|
|
|
|
prompts:
|
|
- text: |
|
|
Hello! Please respond with exactly: "Test infrastructure is working correctly."
|
|
|
|
Do not use any tools. Just respond with that exact text.
|
|
|
|
approvalStrategy:
|
|
type: auto-approve
|
|
|
|
behavior:
|
|
mustNotUseTools: [bash, write, edit, task, read, glob, grep, list]
|
|
minToolCalls: 0
|
|
maxToolCalls: 0
|
|
requiresApproval: false
|
|
expectedAgent: "openagent"
|
|
expectedModel: "opencode/big-pickle"
|
|
expectedResponse:
|
|
contains:
|
|
- "Test infrastructure is working correctly"
|
|
description: "Agent should respond with exact confirmation text"
|
|
|
|
expectedViolations:
|
|
- rule: approval-gate
|
|
shouldViolate: false
|
|
severity: error
|
|
description: "No approval needed for simple response"
|
|
- rule: tool-usage
|
|
shouldViolate: false
|
|
severity: error
|
|
description: "Should not use any tools"
|
|
|
|
timeout: 30000 # 30 seconds should be plenty for a simple response
|
|
|
|
tags:
|
|
- golden
|
|
- smoke-test
|
|
- minimal
|
|
- fast
|