* chore: update package-lock.json after npm install * chore: remove unused ajv-cli, keep vitest at stable 1.6.1 - Removed ajv-cli (unused dev dependency, had high severity vulnerability) - Kept vitest at 1.6.1 (stable, upgrading to v4 requires extensive refactoring) - Reduced vulnerabilities from 6 to 4 (all moderate, dev-only) - All remaining vulnerabilities are in esbuild/vite (dev server only, no production impact) - Build and tests verified working * feat(evals): add execution-balance validation to 21 OpenAgent tests and enhance agent system Major improvements to evaluation framework and agent capabilities: ## Execution Balance Evaluator Integration - Added execution-balance rule to 21 OpenAgent tests (2.7% → 31.5% coverage) - Validates "read before execute" pattern across critical rules, workflows, and edge cases - Tests now check: read operations before execution tools, healthy read/exec ratio (≥1.0) - Coverage includes: approval-gate, context-loading, stop-on-failure, report-first tests ## Agent System Enhancements - Enhanced OpenCoder agent with critical rules enforcement (approval gates, context loading) - Added structured delegation rules and workflow phases - Improved error handling with REPORT→PROPOSE→REQUEST→FIX pattern ## New Agent Creation System - Added research-backed agent creation templates and guides - Includes 8 test templates covering planning, context, incremental execution, tools - Automated agent scaffolding with /create-agent command - Complete documentation in .opencode/command/openagents/new-agents/ ## OpenCoder Test Suite - Added 8 comprehensive test cases for OpenCoder agent - Tests cover: planning, context-loading, delegation, error-handling, implementation - Includes DEBUG_GUIDE.md and QUICK_TEST_GUIDE.md for developers ## Documentation & Tooling - Added DEVELOPMENT.md guide for contributors - Enhanced CONTRIBUTING.md with quick reference and common commands - Improved event-logger with DEBUG_VERBOSE mode for detailed output - Added debug scripts: show-test-conversation.sh, run-test-verbose.sh - Removed deprecated EXTERNAL_PR_GUIDE.md ## Version Bump - Bumped version to 0.0.3 (patch) Files changed: 60+ files (21 test updates, 16 new agent templates, 8 new tests, docs, tooling) * feat(ci): add PR title validation for semantic versioning Add automated PR title validation to ensure conventional commit format. This enables proper automatic version bumping when PRs are merged. Features: - Validates PR titles against conventional commit patterns - Shows expected version bump (major/minor/patch) - Posts helpful comment with examples if validation fails - Supports all conventional commit types (feat, fix, docs, etc.) - Supports breaking changes (feat!, fix!) - Supports pre-release tags ([alpha], [beta], [rc]) When PR titles follow the format, version bumping works correctly: - feat: → minor bump (0.3.0 → 0.4.0) - fix: → patch bump (0.3.0 → 0.3.1) - feat!: → major bump (0.3.0 → 1.0.0) * fix(ci): improve PR checks workflow with sequential execution Reorganize PR checks to run in logical sequence and skip unnecessary checks: Changes: - Run PR title validation first (fast, always required) - Detect changed files to determine which checks to run - Only run build checks if evals/ files changed - Add comprehensive summary job showing all check results - Prevent cache errors when evals files not changed Benefits: - Faster PR checks (skip unnecessary builds) - Clear sequential execution (title → changes → build) - Better error messages and summaries - Reduced CI/CD costs * fix(ci): resolve PR #42 issues and refactor prompt architecture ## Fixed PR #42 Issues 1. **Prompt Validation** - Refactored to new architecture where agent files are canonical defaults 2. **Package-lock.json** - Fixed workflow to use root package-lock (npm workspaces) 3. **Git Merge Base** - Added fetch-depth: 0 to PR checks workflow ## New Prompt Architecture **Before:** - Agent files had to match .opencode/prompts/<agent>/default.md - Caused validation failures when updating prompts - Redundant duplication **After:** - Agent files (.opencode/agent/*.md) = Canonical defaults (source of truth) - Prompt variants (.opencode/prompts/<agent>/<model>.md) = Model-specific optimizations - No more default.md files needed ## Changes ### Workflows - pr-checks.yml: Added fetch-depth: 0 for full git history - validate-test-suites.yml: Fixed package-lock path for npm workspaces - validate-registry.yml: Updated validation messaging ### Scripts - validate-pr.sh: Validates prompt library structure (rejects default.md files) - use-prompt.sh: Handles 'default' as agent file, variants as model-specific - test-prompt.sh: Tests default variant correctly, saves results to prompts/results/ ### Documentation - .opencode/prompts/README.md: Updated architecture explanation - docs/contributing/CONTRIBUTING.md: Updated prompt workflow - scripts/development/demo.sh: Updated structure display ## Results - 12 files changed, 231 insertions(+), 811 deletions(-) - Net reduction: 580 lines (cleaner codebase) - All validations passing - Results directory structure preserved - Backwards compatible with existing test results |
||
|---|---|---|
| .. | ||
| templates | ||
| create-agent.md | ||
| create-tests.md | ||
| README.md | ||
New Agent Creation System
Research-backed agent creation following Anthropic 2025 best practices
Overview
This command system helps you create production-ready OpenCode agents with:
- ✅ Minimal prompts (~500 tokens at "right altitude")
- ✅ Single agent + tools (not multi-agent for coding)
- ✅ Just-in-time context (loaded on demand, not pre-loaded)
- ✅ Clear tool definitions (purpose, when to use, when not to use)
- ✅ Comprehensive testing (8 essential test types)
Quick Start
Create a New Agent
# Interactive agent creation
/create-agent my-agent-name
# Or specify in prompt
"Create a new agent called 'python-dev' for Python development"
Generate Test Suite
# Generate 8 comprehensive tests for existing agent
/create-tests my-agent-name
Research-Backed Principles
1. Single Agent + Tools > Multi-Agent for Coding
Finding: "Most coding tasks involve fewer truly parallelizable tasks than research" (Anthropic 2025)
Why this matters:
- Code changes are deeply dependent on each other
- Sub-agents can't coordinate edits to the same file
- Agents waste time duplicating work
- Multi-agent excels at research (90.2% improvement) because searches are independent
- Code is sequential
Application:
- Use ONE lead agent with tool-based sub-functions
- NOT autonomous sub-agents for coding
- Multi-agent only for truly independent tasks:
- Static analysis (no coordination needed)
- Test execution
- Code search/retrieval
- NOT for: refactoring, architecture decisions, multi-file changes
2. Right Altitude: Minimal Prompts
Finding: "Find the smallest possible set of high-signal tokens that maximize likelihood of desired outcome"
The Balance:
| Too Vague | Right Altitude ✅ | Too Rigid |
|---|---|---|
| "Write good code" | Clear heuristics + examples | 50-line prompt with edge cases |
| Fails to guide behavior | Flexible but specific | Brittle, hard to maintain |
Application:
- System prompt: Minimal (~500 tokens)
- Clear heuristics, not exhaustive rules
- Examples > edge case lists
- Show ONE canonical example, not 20 scenarios
3. Just-in-Time Context Loading
Finding: "Agents discover context layer by layer. File metadata guides behavior. Prevents drowning in irrelevant information"
Context Management Layers:
- System prompt: Minimal (~500 tokens). Clear heuristics, not exhaustive rules.
- Just-in-time retrieval: Tools that agents call to load context on demand (file paths, not full content)
- Working memory: Keep only what's needed for the current task
Why this beats pre-loading:
- Agents discover context layer by layer
- File metadata (size, name, timestamps) guide behavior
- Prevents "drowning in irrelevant information"
4. CLAUDE.md Pattern
Finding: Anthropic's Claude Code uses this in production
Create a project context file automatically loaded into every session:
# Project Context
## Bash Commands
- npm run test: Run unit tests
- npm run lint: Check code style
- npm run typecheck: Check TypeScript
## Code Style
- Use ES modules (import/export)
- Destructure imports when possible
- Use async/await, not callbacks
## Common Files & Patterns
- API handlers in src/handlers/
- Business logic in src/logic/
- Tests mirror source structure
## Workflow Rules
- Always run typecheck before committing
- Don't modify test files when writing implementation
- Use git history to understand WHY, not WHAT
Benefits:
- Eliminates repetitive context-loading
- Shared across team (check into git)
- Tuned like any prompt (run through prompt improvers)
5. Tool Clarity
Finding: "Tool ambiguity is one of the biggest failure modes"
Bad tool design:
tool: "search_code"
description: "search code" # Ambiguous!
Good tool design:
tool: "read_file"
purpose: "Load a specific file for analysis or modification"
when_to_use: "You need to examine or edit a file"
when_not_to_use: "You already have the file content in context"
Key principle: If a human engineer can't definitively say which tool to use, neither can the agent.
6. Extended Thinking for Decomposition
Finding: "Improved instruction-following and reasoning efficiency for complex decomposition"
Before jumping to code, trigger extended thinking:
"Think about how to approach this problem. What files need to change?
What are the dependencies? What should we test?"
Phrases mapped to thinking budget:
- "think" = basic
- "think hard" = 2x budget
- "think harder" = 3x budget
- "ultrathink" = maximum
7. Parallel Tool Calling
Finding: "Parallel tool calling cut research time by up to 90% for complex queries"
Design workflows where agent can call multiple tools simultaneously:
Can do in parallel:
- Run linter
- Execute tests
- Check type errors
NOT in parallel (sequential):
- Apply fix, then test
8. Outcome-Focused Evaluation
Finding: "Token usage explains 80% of performance variance. Number of tool calls ~10%. Model choice ~10%"
What to measure:
- ✅ Does it solve the task?
- ✅ Token usage reasonable?
- ✅ Tool calls appropriate?
- ❌ NOT: "Did it follow exact steps I imagined?"
Application:
- Optimize for using enough tokens to solve the problem
- Don't minimize tool calls (some redundancy is fine)
- Evaluate on real failure cases, not synthetic tests
Agent Structure
Minimal System Prompt Template (~500 tokens)
---
description: "{one-line purpose}"
mode: primary
temperature: 0.1-0.7
tools:
read: true
write: true
edit: true
bash: true
glob: true
grep: true
---
# {Agent Name}
<role>
{Clear, concise role - what this agent does}
</role>
<approach>
1. {First step - usually read/understand}
2. {Second step - usually think/plan}
3. {Third step - usually implement/execute}
4. {Fourth step - usually verify/test}
5. {Fifth step - usually complete/handoff}
</approach>
<heuristics>
- {Key heuristic 1 - how to approach problems}
- {Key heuristic 2 - when to use tools}
- {Key heuristic 3 - how to verify work}
- {Key heuristic 4 - when to stop/report}
</heuristics>
<output>
Always include:
- What you did
- Why you did it that way
- {Domain-specific output requirement}
</output>
<examples>
<example name="{Canonical Use Case}">
**User**: "{typical request}"
**Agent**:
1. {Step 1 with tool usage}
2. {Step 2 with reasoning}
3. {Step 3 with output}
**Result**: {Expected outcome}
</example>
</examples>
Test Suite (8 Essential Tests)
Every agent gets 8 comprehensive tests:
- Planning & Approval - Verify plan-first approach
- Context Loading - Ensure just-in-time context retrieval
- Incremental Implementation - Verify step-by-step execution
- Tool Usage - Check correct tool selection and usage
- Error Handling - Verify stop-on-failure behavior
- Extended Thinking - Check decomposition before coding
- Compaction - Verify summarization for long sessions
- Completion - Check proper output and handoff
What NOT to Do
Based on failure modes found in production:
Don't:
- ❌ Create sub-agents for dependent tasks (code is sequential)
- ❌ Pre-load entire codebase into context (use just-in-time retrieval)
- ❌ Write exhaustive edge case lists in prompts (brittle, hard to maintain)
- ❌ Give vague tool descriptions (major failure mode)
- ❌ Use multi-agent if you could use single agent + tools
- ❌ Hardcode complex logic in prompts (use tools instead)
- ❌ Minimize tool calls (some redundancy is fine)
Do:
- ✅ Let agents discover context via tools
- ✅ Use examples instead of rules
- ✅ Keep system prompt minimal (~500 tokens)
- ✅ Be explicit about effort budgets ("3-5 tool calls, not 50")
- ✅ Evaluate on real failure cases, not synthetic tests
- ✅ Measure outcomes: Does it solve the task?
Files Created
When you create a new agent, the system generates:
.opencode/agent/{agent-name}.md
└─ Minimal system prompt (~500 tokens)
.opencode/context/project/{agent-name}-context.md
└─ Project context (CLAUDE.md pattern)
evals/agents/{agent-name}/
├─ config/
│ └─ config.yaml
└─ tests/
├─ planning/
│ └─ planning-approval-001.yaml
├─ context-loading/
│ └─ context-before-code-001.yaml
├─ implementation/
│ ├─ incremental-001.yaml
│ ├─ tool-usage-001.yaml
│ └─ extended-thinking-001.yaml
├─ error-handling/
│ └─ stop-on-failure-001.yaml
├─ long-horizon/
│ └─ compaction-001.yaml
└─ completion/
└─ handoff-001.yaml
registry.json (updated)
Usage Examples
Example 1: Create Python Development Agent
User: "Create a new agent for Python development with testing and linting"
System creates:
- Agent: python-dev
- System prompt: ~500 tokens
- Tools: read, write, edit, bash, glob, grep
- Context file: Python-specific commands and patterns
- 8 comprehensive tests
Example 2: Create API Testing Agent
User: "Create an agent for API endpoint testing"
System creates:
- Agent: api-tester
- System prompt: ~500 tokens
- Tools: read, bash, glob, grep (no write/edit)
- Context file: API testing patterns and commands
- 8 comprehensive tests
Running Tests
# Run all tests for an agent
cd evals/framework
npm test -- --agent=my-agent-name
# Run specific category
npm test -- --agent=my-agent-name --category=planning
# Run single test
npm test -- --agent=my-agent-name --test=planning-approval-001
Iteration and Improvement
- Test with real use cases (not just synthetic tests)
- Measure outcomes: Does it solve the task?
- Iterate based on actual failures (not imagined edge cases)
- Update status to "stable" when proven in production
Research References
-
Anthropic Multi-Agent Research (Sept-Dec 2025)
- Single agent + tools > multi-agent for coding
- Token usage explains 80% of performance variance
-
Context Engineering Best Practices (Sept 2025)
- "Find the smallest possible set of high-signal tokens"
- Just-in-time retrieval beats pre-loading
-
Claude Code Production Patterns
- CLAUDE.md pattern for project context
- Extended thinking for complex decomposition
- Compaction for long-horizon tasks
Support
For questions or issues:
- Check existing agents:
.opencode/agent/opencoder.md,.opencode/agent/openagent.md - Review test examples:
evals/agents/openagent/tests/ - See research docs:
docs/agents/research-backed-prompt-design.md