* feat(agents): implement category-based agent organization system - Organize agents into domain categories (core, development, content, data, product, learning) - Move 10 agents to category subdirectories with proper git rename tracking - Update 13 subagents with category and type metadata in frontmatter - Add category metadata files (0-category.json) documenting common patterns - Implement local registry fallback in install script for offline development - Add comprehensive validation suite with 15 automated tests (100% pass rate) - Enhance registry validation with duplicate ID and consistency checks - Update eval framework with intelligent path resolution (backward compatible) - Archive legacy eval structure to _archive/ for reference - Update all documentation to reflect category-based structure - Bump version to 0.5.0 with accurate CHANGELOG Technical Details: - 23 agents organized (10 category agents, 13 subagents) - 6 category directories created - Path resolution supports both agent IDs and category paths - Registry schema updated to v2.0.0 - 159 files changed, 872 insertions(+), 10738 deletions(-) BREAKING CHANGE: Agent file paths now use category structure. Update references from .opencode/agent/openagent.md to .opencode/agent/core/openagent.md. Eval framework maintains backward compatibility via path resolution. * refactor(evals): consolidate documentation and enhance test infrastructure - Remove temporary project tracking files (PHASE_5_COMPLETE, PROJECT_COMPLETE, etc.) - Consolidate evaluation framework docs into main README - Enhance test execution with improved logging and multi-prompt support - Move system-builder from core to meta category - Add comprehensive test suites for openagent with organized structure - Create evaluation test structure for all subagents - Clean up archived workflows and redundant documentation - Update registry to reflect new agent organization - Add shared test templates and golden test patterns * feat(ci): add manual trigger support for bot-created PRs in validate-registry workflow - Add pr_number input to workflow_dispatch for manually triggering validation - Fetch PR details dynamically when triggered manually - Support both automatic (PR event) and manual (workflow_dispatch) triggers - Enable validation of bot-created PRs like automated version bumps - Update branch detection and push logic to handle both trigger types - Add documentation explaining how to manually trigger for bot PRs * feat(evals): add explicit context file validation to test framework - Add expectedContextFiles field to test YAML schema for explicit context file specification - Enhance context-loading-evaluator to support both auto-detect and explicit validation modes - Update documentation with comprehensive guide and examples - Clean up archived legacy test structure (90+ old test files) - Add new example tests demonstrating explicit context validation - Backward compatible with existing tests * feat(evals): add multi-agent logging system and performance optimizations Implement comprehensive multi-agent logging and performance improvements for the eval framework. Task 01: Multi-Agent Logging System - Add complete hierarchical logging module (evals/framework/src/logging/) - SessionTracker: tracks parent-child delegation hierarchies - MultiAgentLogger: pretty-prints logs with visual indentation - Formatters: box characters and emoji formatting - 37 passing unit tests (session-tracker, logger, integration) - Integrate with SDK event stream handler - Hook into session.created, message.updated, message.part.updated events - Real-time child session detection via timestamp heuristics - Message deduplication for cleaner output - Enable in debug mode only (<1% performance overhead) - Add demo script and comprehensive documentation Task 02: Performance Optimizations - Reduce grace period from 5s to 2s (67% reduction, 10-20% faster tests) - Add PerformanceMetricsEvaluator for bottleneck identification - Collects tool latencies, inference time, idle time - Provides full performance visibility - Update smoke test to validate delegation (core feature) - Replace simple read test with multi-agent delegation test - Tests both parent and child agent functionality - Validates multi-agent logging system Results: - 37 unit tests passing - Smoke test passing (95/100 score) - Clean hierarchical logging output - 10-20% faster test execution - Full multi-agent visibility Files changed: 16 files, +2678 lines Time saved: ~6-8 days (completed in 6 hours vs 3-5 day estimate) * feat(evals): show child agent execution in non-debug mode - Enable MultiAgentLogger in both debug and non-debug modes - Add verbose parameter to control output level - Non-verbose mode shows concise child session lifecycle: - Child agent started message - Child agent completed with duration - Verbose mode (--debug) shows full delegation hierarchy - Update documentation to reflect new behavior This provides visibility into delegation without overwhelming output, giving confidence that child agents are actually running. * feat(evals): improve delegation testing and behavior validation Major improvements to eval framework: Test Suite: - Add 00-smoke-test.yaml (basic read operation) - Rename 01-smoke-test.yaml → 02-delegation-test.yaml (delegation test) - Add simple-responder test agent for delegation testing - Add debug test: simple-subagent-call.yaml Evaluators: - Add agent-model-evaluator.ts for agent/model validation - Enhance behavior-evaluator.ts with detailed task tool output - Improve delegation-evaluator.ts with better evidence tracking Schema & Execution: - Add expectedAgent and expectedModel to test schema - Improve test executor with better logging and error handling Documentation: - Update evals/README.md with new features and performance improvements - Remove outdated MULTI_AGENT_LOGGING_COMPLETE.md Registry: - Add simple-responder test agent to registry Fixes: - Fix dashboard.sh path resolution These changes provide better visibility into delegation, improved test coverage, and clearer validation of agent behavior. * chore: add GitHub Actions workflow and plugin documentation - Add .github/workflows/evals/run-evaluations.yml for automated eval testing - Add dev/ai-tools/opencode/plugins/Plugin-inspiration.md for plugin development reference
471 lines
12 KiB
Markdown
471 lines
12 KiB
Markdown
# Contributing to OpenAgents
|
|
|
|
Thank you for your interest in contributing! This guide will help you add new components to the registry and understand the repository structure.
|
|
|
|
## 📚 Documentation
|
|
|
|
- **[Development Guide](DEVELOPMENT.md)** - Complete guide for developing on this repo (agents, commands, tools, testing)
|
|
- **[Agent Creation System](../../.opencode/command/openagents/new-agents/README.md)** - ⭐ NEW: Research-backed agent creation with templates
|
|
- **[Code of Conduct](CODE_OF_CONDUCT.md)** - Community guidelines
|
|
- **[Adding Evaluators](ADDING_EVALUATOR.md)** - How to add new test evaluators
|
|
|
|
## Repository Structure
|
|
|
|
```
|
|
opencode-agents/
|
|
├── .opencode/
|
|
│ ├── agent/ # Agents (category-based)
|
|
│ │ ├── core/
|
|
│ │ │ ├── openagent.md # Universal orchestrator
|
|
│ │ │ └── opencoder.md # Development specialist
|
|
│ │ ├── meta/
|
|
│ │ │ └── system-builder.md # System architect
|
|
│ │ ├── development/
|
|
│ │ │ ├── frontend-specialist.md
|
|
│ │ │ └── backend-specialist.md
|
|
│ │ ├── content/
|
|
│ │ │ └── copywriter.md
|
|
│ │ └── subagents/ # Specialized subagents
|
|
│ ├── prompts/ # Prompt library (variants and experiments)
|
|
│ │ ├── core/
|
|
│ │ │ ├── openagent/
|
|
│ │ │ │ ├── gpt.md
|
|
│ │ │ │ └── llama.md
|
|
│ │ │ └── opencoder/
|
|
│ │ │ └── gpt.md
|
|
│ │ └── development/
|
|
│ │ └── frontend-specialist/
|
|
│ │ └── gpt.md
|
|
│ ├── command/ # Slash commands
|
|
│ ├── tool/ # Utility tools
|
|
│ ├── plugin/ # Integrations
|
|
│ └── context/ # Context files
|
|
├── evals/
|
|
│ ├── agents/ # Agent test suites
|
|
│ ├── framework/ # Testing framework
|
|
│ └── results/ # Test results
|
|
├── scripts/
|
|
│ ├── prompts/ # Prompt management
|
|
│ └── tests/ # Test utilities
|
|
└── docs/
|
|
├── agents/ # Agent documentation
|
|
├── contributing/ # Contribution guides
|
|
└── guides/ # User guides
|
|
```
|
|
|
|
> **Note**: Agent files now use category-based organization. Core agents are in `core/`,
|
|
> development specialists in `development/`, etc. When referencing agents, use the
|
|
> category prefix: `core/openagent`, `development/frontend-specialist`.
|
|
|
|
### Key Directories
|
|
|
|
- **`.opencode/agent/core/`** - Core system agents (openagent.md, opencoder.md)
|
|
- **`.opencode/agent/meta/`** - Meta-level agents (system-builder.md)
|
|
- **`.opencode/agent/development/`** - Development specialist agents
|
|
- **`.opencode/agent/content/`** - Content creation agents
|
|
- **`.opencode/agent/subagents/`** - Specialized subagents
|
|
- **`.opencode/prompts/`** - Library of prompt variants for different models (category-based)
|
|
- **`evals/`** - Testing framework and test suites
|
|
- **`scripts/`** - Automation and utility scripts
|
|
- **`docs/`** - Documentation and guides
|
|
|
|
## Quick Start
|
|
|
|
1. Fork the repository
|
|
2. Create a new branch for your feature
|
|
3. Add your component
|
|
4. Test it works
|
|
5. Submit a pull request
|
|
|
|
## Adding New Components
|
|
|
|
### Component Types
|
|
|
|
- **Agents** (`.opencode/agent/*.md`) - Main AI agents
|
|
- **Subagents** (`.opencode/agent/subagents/*.md`) - Specialized helpers
|
|
- **Commands** (`.opencode/command/*.md`) - Slash commands
|
|
- **Tools** (`.opencode/tool/*/index.ts`) - Utility tools
|
|
- **Plugins** (`.opencode/plugin/*.ts`) - Integrations
|
|
- **Contexts** (`.opencode/context/**/*.md`) - Context files
|
|
|
|
### Component Structure
|
|
|
|
#### For Markdown Files (Agents, Commands, Contexts)
|
|
|
|
All markdown files should include YAML frontmatter:
|
|
|
|
```markdown
|
|
---
|
|
description: "Brief description of what this does"
|
|
mode: primary # For agents only
|
|
temperature: 0.1 # Optional - for agents only
|
|
tools: # For agents only
|
|
read: true
|
|
edit: true
|
|
write: true
|
|
permissions: # Optional
|
|
bash:
|
|
"*": "deny"
|
|
---
|
|
|
|
# Component Name
|
|
|
|
Your component content here...
|
|
```
|
|
|
|
**Required fields:**
|
|
- `description` - Brief description (all components)
|
|
|
|
**Agent-specific fields:**
|
|
- `mode` - Agent mode (primary, secondary, etc.)
|
|
- `model` - AI model to use
|
|
- `temperature` - Temperature setting
|
|
- `tools` - Available tools
|
|
- `permissions` - Security permissions
|
|
|
|
#### For TypeScript Files (Tools, Plugins)
|
|
|
|
Include JSDoc comments at the top:
|
|
|
|
```typescript
|
|
/**
|
|
* Tool Name
|
|
*
|
|
* Brief description of what this tool does
|
|
*/
|
|
|
|
export function myTool() {
|
|
// Implementation
|
|
}
|
|
```
|
|
|
|
### File Naming Conventions
|
|
|
|
- **kebab-case** for file names: `my-new-agent.md`
|
|
- **PascalCase** for TypeScript types/interfaces
|
|
- **camelCase** for variables and functions
|
|
|
|
### Adding Your Component
|
|
|
|
1. **Create the component file** in the appropriate directory:
|
|
```bash
|
|
# Example: Adding a new agent
|
|
touch .opencode/agent/my-new-agent.md
|
|
```
|
|
|
|
2. **Add frontmatter and content** following the structure above
|
|
|
|
3. **Test your component**:
|
|
```bash
|
|
# Validate structure
|
|
./scripts/registry/validate-component.sh
|
|
```
|
|
|
|
4. **Update the registry** (automatic on merge to main):
|
|
```bash
|
|
# Manual update (optional)
|
|
./scripts/registry/register-component.sh
|
|
```
|
|
|
|
## Component Categories
|
|
|
|
When adding components, they're automatically categorized:
|
|
|
|
- **core** - Essential components included in minimal installs
|
|
- **extended** - Additional features for developer profile
|
|
- **advanced** - Experimental or specialized components
|
|
|
|
The auto-registration script assigns categories based on component type and location.
|
|
|
|
## Testing Your Component
|
|
|
|
### Local Testing
|
|
|
|
1. **Install locally**:
|
|
```bash
|
|
# Test the installer
|
|
./install.sh --list
|
|
```
|
|
|
|
2. **Validate structure**:
|
|
```bash
|
|
./scripts/registry/validate-component.sh
|
|
```
|
|
|
|
3. **Test with OpenCode**:
|
|
```bash
|
|
opencode --agent your-new-agent
|
|
```
|
|
|
|
### Automated Testing
|
|
|
|
When you submit a PR, GitHub Actions will:
|
|
- Validate component structure
|
|
- Validate prompts use defaults
|
|
- Update the registry
|
|
- Run validation checks
|
|
|
|
**Important**: PRs will fail if agents don't use their default prompts. This ensures the main branch stays stable.
|
|
|
|
## Prompt Library System
|
|
|
|
OpenCode uses a model-specific prompt library to support different AI models while keeping the main branch stable.
|
|
|
|
### How It Works
|
|
|
|
```
|
|
.opencode/
|
|
├── agent/ # Active prompts (always default in PRs)
|
|
│ ├── openagent.md
|
|
│ └── opencoder.md
|
|
└── prompts/ # Prompt library (model-specific variants)
|
|
├── openagent/
|
|
│ ├── gpt.md # GPT-4 optimized
|
|
│ ├── gemini.md # Gemini optimized
|
|
│ ├── grok.md # Grok optimized
|
|
│ ├── llama.md # Llama/OSS optimized
|
|
│ ├── TEMPLATE.md # Template for new variants
|
|
│ ├── README.md # Capabilities table
|
|
│ └── results/ # Test results (all variants)
|
|
└── opencoder/
|
|
└── ...
|
|
|
|
**Architecture:**
|
|
- Agent files (`.opencode/agent/*.md`) = Canonical defaults
|
|
- Prompt variants (`.opencode/prompts/<agent>/<model>.md`) = Model-specific optimizations
|
|
```
|
|
|
|
### For Contributors
|
|
|
|
#### Testing a Prompt Variant
|
|
|
|
```bash
|
|
# Test a specific variant
|
|
./scripts/prompts/test-prompt.sh core/openagent sonnet-4
|
|
|
|
# View results
|
|
cat .opencode/prompts/core/openagent/results/sonnet-4-results.json
|
|
```
|
|
|
|
#### Creating a New Variant
|
|
|
|
1. **Copy the template:**
|
|
```bash
|
|
cp .opencode/prompts/core/openagent/TEMPLATE.md .opencode/prompts/core/openagent/my-variant.md
|
|
```
|
|
|
|
2. **Edit your variant:**
|
|
- Add variant info (target model, focus, author)
|
|
- Document changes from default
|
|
- Write your prompt
|
|
|
|
3. **Test it:**
|
|
```bash
|
|
./scripts/prompts/test-prompt.sh core/openagent my-variant
|
|
```
|
|
|
|
4. **Update the README:**
|
|
- Add your variant to the capabilities table in `.opencode/prompts/core/openagent/README.md`
|
|
- Document test results
|
|
- Explain what it optimizes for
|
|
|
|
5. **Submit PR:**
|
|
- Include your variant file (e.g., `my-variant.md`)
|
|
- Include updated README with results
|
|
- **Do NOT change the default prompt**
|
|
- **Do NOT change `.opencode/agent/core/openagent.md`**
|
|
|
|
#### PR Requirements for Prompts
|
|
|
|
**All PRs must use default prompts.** CI automatically validates this.
|
|
|
|
Before submitting a PR:
|
|
```bash
|
|
# Ensure you're using defaults
|
|
./scripts/prompts/validate-pr.sh
|
|
|
|
# If validation fails, restore defaults
|
|
./scripts/prompts/use-prompt.sh core/openagent default
|
|
./scripts/prompts/use-prompt.sh core/opencoder default
|
|
```
|
|
|
|
#### Why This System?
|
|
|
|
- **Stability**: Main branch always uses tested defaults
|
|
- **Experimentation**: Contributors can optimize for specific models
|
|
- **Transparency**: Test results are documented for each variant
|
|
- **Flexibility**: Users can choose the best prompt for their model
|
|
|
|
### For Maintainers
|
|
|
|
#### Promoting a Variant to Default
|
|
|
|
When a variant proves superior:
|
|
|
|
1. **Verify test results:**
|
|
```bash
|
|
cat .opencode/prompts/core/openagent/results/variant-results.json
|
|
```
|
|
|
|
2. **Update agent file (canonical default):**
|
|
```bash
|
|
cp .opencode/prompts/core/openagent/variant.md .opencode/agent/core/openagent.md
|
|
```
|
|
|
|
3. **Update capabilities table** in README
|
|
|
|
4. **Commit with clear message:**
|
|
```bash
|
|
git add .opencode/agent/core/openagent.md
|
|
git commit -m "feat(openagent): promote variant to default - improved X by Y%"
|
|
```
|
|
|
|
**Note:** In the new architecture, agent files are the canonical defaults. There are no `default.md` files in the prompts directory.
|
|
|
|
## Pull Request Guidelines
|
|
|
|
### PR Title Format
|
|
|
|
Use conventional commits:
|
|
- `feat: add new agent for X`
|
|
- `fix: correct issue in Y command`
|
|
- `docs: update Z documentation`
|
|
- `chore: update dependencies`
|
|
- `prompt: add new variant for X model`
|
|
|
|
### PR Description
|
|
|
|
Include:
|
|
1. **What** - What component are you adding/changing?
|
|
2. **Why** - Why is this useful?
|
|
3. **How** - How does it work?
|
|
4. **Testing** - How did you test it?
|
|
|
|
Example:
|
|
```markdown
|
|
## What
|
|
Adds a new `database-agent` for managing database migrations.
|
|
|
|
## Why
|
|
Automates common database tasks and ensures migration safety.
|
|
|
|
## How
|
|
- Scans migration files
|
|
- Validates migration order
|
|
- Runs migrations with rollback support
|
|
|
|
## Testing
|
|
- [x] Validated with `./scripts/registry/validate-component.sh`
|
|
- [x] Tested with PostgreSQL and MySQL
|
|
- [x] Tested rollback scenarios
|
|
```
|
|
|
|
## Component Dependencies
|
|
|
|
If your component depends on others, declare them in the registry:
|
|
|
|
```json
|
|
{
|
|
"id": "my-component",
|
|
"dependencies": ["tool:env", "agent:task-manager"]
|
|
}
|
|
```
|
|
|
|
The installer will automatically install dependencies.
|
|
|
|
## Registry Auto-Update
|
|
|
|
The registry is automatically updated when:
|
|
- You push to `main` branch
|
|
- Changes are made to `.opencode/` directory
|
|
|
|
The GitHub Action:
|
|
1. Scans all components
|
|
2. Extracts metadata
|
|
3. Updates `registry.json`
|
|
4. Commits changes
|
|
|
|
You don't need to manually edit `registry.json`!
|
|
|
|
## Code Style
|
|
|
|
### Markdown
|
|
- Use clear, concise language
|
|
- Include examples
|
|
- Add code blocks with syntax highlighting
|
|
- Use proper heading hierarchy
|
|
|
|
### TypeScript
|
|
- Follow existing code style
|
|
- Add JSDoc comments
|
|
- Use TypeScript types (no `any`)
|
|
- Export functions explicitly
|
|
|
|
### Bash Scripts
|
|
- Use `set -e` for error handling
|
|
- Add comments for complex logic
|
|
- Use meaningful variable names
|
|
- Include help text
|
|
|
|
## Quick Reference
|
|
|
|
### Creating a New Agent
|
|
|
|
```bash
|
|
# Use the automated system (recommended)
|
|
/create-agent my-agent-name
|
|
|
|
# Or manually follow the development guide
|
|
# See: docs/contributing/DEVELOPMENT.md#creating-new-agents
|
|
```
|
|
|
|
### Running Tests
|
|
|
|
```bash
|
|
cd evals/framework
|
|
npm test -- --agent=my-agent
|
|
```
|
|
|
|
### Validating Before PR
|
|
|
|
```bash
|
|
# Validate structure
|
|
./scripts/registry/validate-component.sh
|
|
|
|
# Ensure using defaults
|
|
./scripts/prompts/validate-pr.sh
|
|
|
|
# Run tests
|
|
cd evals/framework && npm test
|
|
```
|
|
|
|
### Common Commands
|
|
|
|
```bash
|
|
# List available components
|
|
./install.sh --list
|
|
|
|
# Validate registry
|
|
make validate-registry
|
|
|
|
# Update registry
|
|
make update-registry
|
|
|
|
# Test a prompt variant
|
|
./scripts/prompts/test-prompt.sh openagent my-variant
|
|
```
|
|
|
|
## Questions?
|
|
|
|
- **Issues**: Open an issue for bugs or feature requests
|
|
- **Discussions**: Use GitHub Discussions for questions
|
|
- **Security**: Email security issues privately
|
|
- **Development Help**: See [DEVELOPMENT.md](DEVELOPMENT.md) for detailed guides
|
|
|
|
## License
|
|
|
|
By contributing, you agree that your contributions will be licensed under the MIT License.
|
|
|
|
---
|
|
|
|
Thank you for contributing! 🎉
|