* feat(agents): implement category-based agent organization system - Organize agents into domain categories (core, development, content, data, product, learning) - Move 10 agents to category subdirectories with proper git rename tracking - Update 13 subagents with category and type metadata in frontmatter - Add category metadata files (0-category.json) documenting common patterns - Implement local registry fallback in install script for offline development - Add comprehensive validation suite with 15 automated tests (100% pass rate) - Enhance registry validation with duplicate ID and consistency checks - Update eval framework with intelligent path resolution (backward compatible) - Archive legacy eval structure to _archive/ for reference - Update all documentation to reflect category-based structure - Bump version to 0.5.0 with accurate CHANGELOG Technical Details: - 23 agents organized (10 category agents, 13 subagents) - 6 category directories created - Path resolution supports both agent IDs and category paths - Registry schema updated to v2.0.0 - 159 files changed, 872 insertions(+), 10738 deletions(-) BREAKING CHANGE: Agent file paths now use category structure. Update references from .opencode/agent/openagent.md to .opencode/agent/core/openagent.md. Eval framework maintains backward compatibility via path resolution. * refactor(evals): consolidate documentation and enhance test infrastructure - Remove temporary project tracking files (PHASE_5_COMPLETE, PROJECT_COMPLETE, etc.) - Consolidate evaluation framework docs into main README - Enhance test execution with improved logging and multi-prompt support - Move system-builder from core to meta category - Add comprehensive test suites for openagent with organized structure - Create evaluation test structure for all subagents - Clean up archived workflows and redundant documentation - Update registry to reflect new agent organization - Add shared test templates and golden test patterns * feat(ci): add manual trigger support for bot-created PRs in validate-registry workflow - Add pr_number input to workflow_dispatch for manually triggering validation - Fetch PR details dynamically when triggered manually - Support both automatic (PR event) and manual (workflow_dispatch) triggers - Enable validation of bot-created PRs like automated version bumps - Update branch detection and push logic to handle both trigger types - Add documentation explaining how to manually trigger for bot PRs * feat(evals): add explicit context file validation to test framework - Add expectedContextFiles field to test YAML schema for explicit context file specification - Enhance context-loading-evaluator to support both auto-detect and explicit validation modes - Update documentation with comprehensive guide and examples - Clean up archived legacy test structure (90+ old test files) - Add new example tests demonstrating explicit context validation - Backward compatible with existing tests * feat(evals): add multi-agent logging system and performance optimizations Implement comprehensive multi-agent logging and performance improvements for the eval framework. Task 01: Multi-Agent Logging System - Add complete hierarchical logging module (evals/framework/src/logging/) - SessionTracker: tracks parent-child delegation hierarchies - MultiAgentLogger: pretty-prints logs with visual indentation - Formatters: box characters and emoji formatting - 37 passing unit tests (session-tracker, logger, integration) - Integrate with SDK event stream handler - Hook into session.created, message.updated, message.part.updated events - Real-time child session detection via timestamp heuristics - Message deduplication for cleaner output - Enable in debug mode only (<1% performance overhead) - Add demo script and comprehensive documentation Task 02: Performance Optimizations - Reduce grace period from 5s to 2s (67% reduction, 10-20% faster tests) - Add PerformanceMetricsEvaluator for bottleneck identification - Collects tool latencies, inference time, idle time - Provides full performance visibility - Update smoke test to validate delegation (core feature) - Replace simple read test with multi-agent delegation test - Tests both parent and child agent functionality - Validates multi-agent logging system Results: - 37 unit tests passing - Smoke test passing (95/100 score) - Clean hierarchical logging output - 10-20% faster test execution - Full multi-agent visibility Files changed: 16 files, +2678 lines Time saved: ~6-8 days (completed in 6 hours vs 3-5 day estimate) * feat(evals): show child agent execution in non-debug mode - Enable MultiAgentLogger in both debug and non-debug modes - Add verbose parameter to control output level - Non-verbose mode shows concise child session lifecycle: - Child agent started message - Child agent completed with duration - Verbose mode (--debug) shows full delegation hierarchy - Update documentation to reflect new behavior This provides visibility into delegation without overwhelming output, giving confidence that child agents are actually running. * feat(evals): improve delegation testing and behavior validation Major improvements to eval framework: Test Suite: - Add 00-smoke-test.yaml (basic read operation) - Rename 01-smoke-test.yaml → 02-delegation-test.yaml (delegation test) - Add simple-responder test agent for delegation testing - Add debug test: simple-subagent-call.yaml Evaluators: - Add agent-model-evaluator.ts for agent/model validation - Enhance behavior-evaluator.ts with detailed task tool output - Improve delegation-evaluator.ts with better evidence tracking Schema & Execution: - Add expectedAgent and expectedModel to test schema - Improve test executor with better logging and error handling Documentation: - Update evals/README.md with new features and performance improvements - Remove outdated MULTI_AGENT_LOGGING_COMPLETE.md Registry: - Add simple-responder test agent to registry Fixes: - Fix dashboard.sh path resolution These changes provide better visibility into delegation, improved test coverage, and clearer validation of agent behavior. * chore: add GitHub Actions workflow and plugin documentation - Add .github/workflows/evals/run-evaluations.yml for automated eval testing - Add dev/ai-tools/opencode/plugins/Plugin-inspiration.md for plugin development reference
12 KiB
Contributing to OpenAgents
Thank you for your interest in contributing! This guide will help you add new components to the registry and understand the repository structure.
📚 Documentation
- Development Guide - Complete guide for developing on this repo (agents, commands, tools, testing)
- Agent Creation System - ⭐ NEW: Research-backed agent creation with templates
- Code of Conduct - Community guidelines
- Adding Evaluators - How to add new test evaluators
Repository Structure
opencode-agents/
├── .opencode/
│ ├── agent/ # Agents (category-based)
│ │ ├── core/
│ │ │ ├── openagent.md # Universal orchestrator
│ │ │ └── opencoder.md # Development specialist
│ │ ├── meta/
│ │ │ └── system-builder.md # System architect
│ │ ├── development/
│ │ │ ├── frontend-specialist.md
│ │ │ └── backend-specialist.md
│ │ ├── content/
│ │ │ └── copywriter.md
│ │ └── subagents/ # Specialized subagents
│ ├── prompts/ # Prompt library (variants and experiments)
│ │ ├── core/
│ │ │ ├── openagent/
│ │ │ │ ├── gpt.md
│ │ │ │ └── llama.md
│ │ │ └── opencoder/
│ │ │ └── gpt.md
│ │ └── development/
│ │ └── frontend-specialist/
│ │ └── gpt.md
│ ├── command/ # Slash commands
│ ├── tool/ # Utility tools
│ ├── plugin/ # Integrations
│ └── context/ # Context files
├── evals/
│ ├── agents/ # Agent test suites
│ ├── framework/ # Testing framework
│ └── results/ # Test results
├── scripts/
│ ├── prompts/ # Prompt management
│ └── tests/ # Test utilities
└── docs/
├── agents/ # Agent documentation
├── contributing/ # Contribution guides
└── guides/ # User guides
Note
: Agent files now use category-based organization. Core agents are in
core/, development specialists indevelopment/, etc. When referencing agents, use the category prefix:core/openagent,development/frontend-specialist.
Key Directories
.opencode/agent/core/- Core system agents (openagent.md, opencoder.md).opencode/agent/meta/- Meta-level agents (system-builder.md).opencode/agent/development/- Development specialist agents.opencode/agent/content/- Content creation agents.opencode/agent/subagents/- Specialized subagents.opencode/prompts/- Library of prompt variants for different models (category-based)evals/- Testing framework and test suitesscripts/- Automation and utility scriptsdocs/- Documentation and guides
Quick Start
- Fork the repository
- Create a new branch for your feature
- Add your component
- Test it works
- Submit a pull request
Adding New Components
Component Types
- Agents (
.opencode/agent/*.md) - Main AI agents - Subagents (
.opencode/agent/subagents/*.md) - Specialized helpers - Commands (
.opencode/command/*.md) - Slash commands - Tools (
.opencode/tool/*/index.ts) - Utility tools - Plugins (
.opencode/plugin/*.ts) - Integrations - Contexts (
.opencode/context/**/*.md) - Context files
Component Structure
For Markdown Files (Agents, Commands, Contexts)
All markdown files should include YAML frontmatter:
---
description: "Brief description of what this does"
mode: primary # For agents only
temperature: 0.1 # Optional - for agents only
tools: # For agents only
read: true
edit: true
write: true
permissions: # Optional
bash:
"*": "deny"
---
# Component Name
Your component content here...
Required fields:
description- Brief description (all components)
Agent-specific fields:
mode- Agent mode (primary, secondary, etc.)model- AI model to usetemperature- Temperature settingtools- Available toolspermissions- Security permissions
For TypeScript Files (Tools, Plugins)
Include JSDoc comments at the top:
/**
* Tool Name
*
* Brief description of what this tool does
*/
export function myTool() {
// Implementation
}
File Naming Conventions
- kebab-case for file names:
my-new-agent.md - PascalCase for TypeScript types/interfaces
- camelCase for variables and functions
Adding Your Component
-
Create the component file in the appropriate directory:
# Example: Adding a new agent touch .opencode/agent/my-new-agent.md -
Add frontmatter and content following the structure above
-
Test your component:
# Validate structure ./scripts/registry/validate-component.sh -
Update the registry (automatic on merge to main):
# Manual update (optional) ./scripts/registry/register-component.sh
Component Categories
When adding components, they're automatically categorized:
- core - Essential components included in minimal installs
- extended - Additional features for developer profile
- advanced - Experimental or specialized components
The auto-registration script assigns categories based on component type and location.
Testing Your Component
Local Testing
-
Install locally:
# Test the installer ./install.sh --list -
Validate structure:
./scripts/registry/validate-component.sh -
Test with OpenCode:
opencode --agent your-new-agent
Automated Testing
When you submit a PR, GitHub Actions will:
- Validate component structure
- Validate prompts use defaults
- Update the registry
- Run validation checks
Important: PRs will fail if agents don't use their default prompts. This ensures the main branch stays stable.
Prompt Library System
OpenCode uses a model-specific prompt library to support different AI models while keeping the main branch stable.
How It Works
.opencode/
├── agent/ # Active prompts (always default in PRs)
│ ├── openagent.md
│ └── opencoder.md
└── prompts/ # Prompt library (model-specific variants)
├── openagent/
│ ├── gpt.md # GPT-4 optimized
│ ├── gemini.md # Gemini optimized
│ ├── grok.md # Grok optimized
│ ├── llama.md # Llama/OSS optimized
│ ├── TEMPLATE.md # Template for new variants
│ ├── README.md # Capabilities table
│ └── results/ # Test results (all variants)
└── opencoder/
└── ...
**Architecture:**
- Agent files (`.opencode/agent/*.md`) = Canonical defaults
- Prompt variants (`.opencode/prompts/<agent>/<model>.md`) = Model-specific optimizations
For Contributors
Testing a Prompt Variant
# Test a specific variant
./scripts/prompts/test-prompt.sh core/openagent sonnet-4
# View results
cat .opencode/prompts/core/openagent/results/sonnet-4-results.json
Creating a New Variant
-
Copy the template:
cp .opencode/prompts/core/openagent/TEMPLATE.md .opencode/prompts/core/openagent/my-variant.md -
Edit your variant:
- Add variant info (target model, focus, author)
- Document changes from default
- Write your prompt
-
Test it:
./scripts/prompts/test-prompt.sh core/openagent my-variant -
Update the README:
- Add your variant to the capabilities table in
.opencode/prompts/core/openagent/README.md - Document test results
- Explain what it optimizes for
- Add your variant to the capabilities table in
-
Submit PR:
- Include your variant file (e.g.,
my-variant.md) - Include updated README with results
- Do NOT change the default prompt
- Do NOT change
.opencode/agent/core/openagent.md
- Include your variant file (e.g.,
PR Requirements for Prompts
All PRs must use default prompts. CI automatically validates this.
Before submitting a PR:
# Ensure you're using defaults
./scripts/prompts/validate-pr.sh
# If validation fails, restore defaults
./scripts/prompts/use-prompt.sh core/openagent default
./scripts/prompts/use-prompt.sh core/opencoder default
Why This System?
- Stability: Main branch always uses tested defaults
- Experimentation: Contributors can optimize for specific models
- Transparency: Test results are documented for each variant
- Flexibility: Users can choose the best prompt for their model
For Maintainers
Promoting a Variant to Default
When a variant proves superior:
-
Verify test results:
cat .opencode/prompts/core/openagent/results/variant-results.json -
Update agent file (canonical default):
cp .opencode/prompts/core/openagent/variant.md .opencode/agent/core/openagent.md -
Update capabilities table in README
-
Commit with clear message:
git add .opencode/agent/core/openagent.md git commit -m "feat(openagent): promote variant to default - improved X by Y%"
Note: In the new architecture, agent files are the canonical defaults. There are no default.md files in the prompts directory.
Pull Request Guidelines
PR Title Format
Use conventional commits:
feat: add new agent for Xfix: correct issue in Y commanddocs: update Z documentationchore: update dependenciesprompt: add new variant for X model
PR Description
Include:
- What - What component are you adding/changing?
- Why - Why is this useful?
- How - How does it work?
- Testing - How did you test it?
Example:
## What
Adds a new `database-agent` for managing database migrations.
## Why
Automates common database tasks and ensures migration safety.
## How
- Scans migration files
- Validates migration order
- Runs migrations with rollback support
## Testing
- [x] Validated with `./scripts/registry/validate-component.sh`
- [x] Tested with PostgreSQL and MySQL
- [x] Tested rollback scenarios
Component Dependencies
If your component depends on others, declare them in the registry:
{
"id": "my-component",
"dependencies": ["tool:env", "agent:task-manager"]
}
The installer will automatically install dependencies.
Registry Auto-Update
The registry is automatically updated when:
- You push to
mainbranch - Changes are made to
.opencode/directory
The GitHub Action:
- Scans all components
- Extracts metadata
- Updates
registry.json - Commits changes
You don't need to manually edit registry.json!
Code Style
Markdown
- Use clear, concise language
- Include examples
- Add code blocks with syntax highlighting
- Use proper heading hierarchy
TypeScript
- Follow existing code style
- Add JSDoc comments
- Use TypeScript types (no
any) - Export functions explicitly
Bash Scripts
- Use
set -efor error handling - Add comments for complex logic
- Use meaningful variable names
- Include help text
Quick Reference
Creating a New Agent
# Use the automated system (recommended)
/create-agent my-agent-name
# Or manually follow the development guide
# See: docs/contributing/DEVELOPMENT.md#creating-new-agents
Running Tests
cd evals/framework
npm test -- --agent=my-agent
Validating Before PR
# Validate structure
./scripts/registry/validate-component.sh
# Ensure using defaults
./scripts/prompts/validate-pr.sh
# Run tests
cd evals/framework && npm test
Common Commands
# List available components
./install.sh --list
# Validate registry
make validate-registry
# Update registry
make update-registry
# Test a prompt variant
./scripts/prompts/test-prompt.sh openagent my-variant
Questions?
- Issues: Open an issue for bugs or feature requests
- Discussions: Use GitHub Discussions for questions
- Security: Email security issues privately
- Development Help: See DEVELOPMENT.md for detailed guides
License
By contributing, you agree that your contributions will be licensed under the MIT License.
Thank you for contributing! 🎉