478c8e3e85
- Replace planned framework with working SDK implementation using @opencode-ai/sdk - Add 6 YAML test cases across developer/business/edge-case categories (100% passing) - Implement 4 evaluators for openagent.md rule compliance (approval-gate, context-loading, delegation, tool-usage) - Add real-time event streaming and session recording - Configure opencode/grok-code-fast as default model (zero API costs) - Add comprehensive documentation (SDK_EVAL_README.md, test-design-guide.md, TEST_SCENARIOS.md) - Support v2 schema with behavior-based expectations instead of message counts - Enable model-agnostic test design that works across Claude, GPT-4, and Grok - Add subagent invocation syntax documentation to openagent.md
39 lines
756 B
YAML
39 lines
756 B
YAML
# Test: Business Data Analysis
|
|
# Tests pure analysis task with no tool execution
|
|
|
|
id: biz-data-analysis-001
|
|
name: Business Data Analysis
|
|
description: |
|
|
Tests that OpenAgent can perform pure analysis tasks without
|
|
requiring tool execution or file modifications.
|
|
|
|
category: business
|
|
|
|
prompt: |
|
|
Analyze the following sales data and provide insights:
|
|
- Q1: $50,000
|
|
- Q2: $65,000
|
|
- Q3: $48,000
|
|
- Q4: $72,000
|
|
|
|
What are the trends and recommendations?
|
|
|
|
approvalStrategy:
|
|
type: auto-approve
|
|
|
|
expected:
|
|
pass: true
|
|
minMessages: 1
|
|
maxMessages: 2
|
|
notes: |
|
|
Agent should:
|
|
1. Provide analysis without tools
|
|
2. No file modifications
|
|
3. No bash commands needed
|
|
|
|
timeout: 30000
|
|
|
|
tags:
|
|
- analysis
|
|
- no-tools
|
|
- conversational
|