OpenWriterControle/.opencode/command/commit-openagents.md
Darren Hinde c8f7103cb6
refactor(evals): consolidate documentation and enhance test infrastructure (#56)
* feat(agents): implement category-based agent organization system

- Organize agents into domain categories (core, development, content, data, product, learning)
- Move 10 agents to category subdirectories with proper git rename tracking
- Update 13 subagents with category and type metadata in frontmatter
- Add category metadata files (0-category.json) documenting common patterns
- Implement local registry fallback in install script for offline development
- Add comprehensive validation suite with 15 automated tests (100% pass rate)
- Enhance registry validation with duplicate ID and consistency checks
- Update eval framework with intelligent path resolution (backward compatible)
- Archive legacy eval structure to _archive/ for reference
- Update all documentation to reflect category-based structure
- Bump version to 0.5.0 with accurate CHANGELOG

Technical Details:
- 23 agents organized (10 category agents, 13 subagents)
- 6 category directories created
- Path resolution supports both agent IDs and category paths
- Registry schema updated to v2.0.0
- 159 files changed, 872 insertions(+), 10738 deletions(-)

BREAKING CHANGE: Agent file paths now use category structure. Update references from .opencode/agent/openagent.md to .opencode/agent/core/openagent.md. Eval framework maintains backward compatibility via path resolution.

* refactor(evals): consolidate documentation and enhance test infrastructure

- Remove temporary project tracking files (PHASE_5_COMPLETE, PROJECT_COMPLETE, etc.)
- Consolidate evaluation framework docs into main README
- Enhance test execution with improved logging and multi-prompt support
- Move system-builder from core to meta category
- Add comprehensive test suites for openagent with organized structure
- Create evaluation test structure for all subagents
- Clean up archived workflows and redundant documentation
- Update registry to reflect new agent organization
- Add shared test templates and golden test patterns

* feat(ci): add manual trigger support for bot-created PRs in validate-registry workflow

- Add pr_number input to workflow_dispatch for manually triggering validation
- Fetch PR details dynamically when triggered manually
- Support both automatic (PR event) and manual (workflow_dispatch) triggers
- Enable validation of bot-created PRs like automated version bumps
- Update branch detection and push logic to handle both trigger types
- Add documentation explaining how to manually trigger for bot PRs

* feat(evals): add explicit context file validation to test framework

- Add expectedContextFiles field to test YAML schema for explicit context file specification
- Enhance context-loading-evaluator to support both auto-detect and explicit validation modes
- Update documentation with comprehensive guide and examples
- Clean up archived legacy test structure (90+ old test files)
- Add new example tests demonstrating explicit context validation
- Backward compatible with existing tests

* feat(evals): add multi-agent logging system and performance optimizations

Implement comprehensive multi-agent logging and performance improvements for the eval framework.

Task 01: Multi-Agent Logging System
- Add complete hierarchical logging module (evals/framework/src/logging/)
  - SessionTracker: tracks parent-child delegation hierarchies
  - MultiAgentLogger: pretty-prints logs with visual indentation
  - Formatters: box characters and emoji formatting
  - 37 passing unit tests (session-tracker, logger, integration)
- Integrate with SDK event stream handler
  - Hook into session.created, message.updated, message.part.updated events
  - Real-time child session detection via timestamp heuristics
  - Message deduplication for cleaner output
- Enable in debug mode only (<1% performance overhead)
- Add demo script and comprehensive documentation

Task 02: Performance Optimizations
- Reduce grace period from 5s to 2s (67% reduction, 10-20% faster tests)
- Add PerformanceMetricsEvaluator for bottleneck identification
  - Collects tool latencies, inference time, idle time
  - Provides full performance visibility
- Update smoke test to validate delegation (core feature)
  - Replace simple read test with multi-agent delegation test
  - Tests both parent and child agent functionality
  - Validates multi-agent logging system

Results:
- 37 unit tests passing
- Smoke test passing (95/100 score)
- Clean hierarchical logging output
- 10-20% faster test execution
- Full multi-agent visibility

Files changed: 16 files, +2678 lines
Time saved: ~6-8 days (completed in 6 hours vs 3-5 day estimate)

* feat(evals): show child agent execution in non-debug mode

- Enable MultiAgentLogger in both debug and non-debug modes
- Add verbose parameter to control output level
- Non-verbose mode shows concise child session lifecycle:
  - Child agent started message
  - Child agent completed with duration
- Verbose mode (--debug) shows full delegation hierarchy
- Update documentation to reflect new behavior

This provides visibility into delegation without overwhelming output,
giving confidence that child agents are actually running.

* feat(evals): improve delegation testing and behavior validation

Major improvements to eval framework:

Test Suite:
- Add 00-smoke-test.yaml (basic read operation)
- Rename 01-smoke-test.yaml → 02-delegation-test.yaml (delegation test)
- Add simple-responder test agent for delegation testing
- Add debug test: simple-subagent-call.yaml

Evaluators:
- Add agent-model-evaluator.ts for agent/model validation
- Enhance behavior-evaluator.ts with detailed task tool output
- Improve delegation-evaluator.ts with better evidence tracking

Schema & Execution:
- Add expectedAgent and expectedModel to test schema
- Improve test executor with better logging and error handling

Documentation:
- Update evals/README.md with new features and performance improvements
- Remove outdated MULTI_AGENT_LOGGING_COMPLETE.md

Registry:
- Add simple-responder test agent to registry

Fixes:
- Fix dashboard.sh path resolution

These changes provide better visibility into delegation, improved test
coverage, and clearer validation of agent behavior.

* chore: add GitHub Actions workflow and plugin documentation

- Add .github/workflows/evals/run-evaluations.yml for automated eval testing
- Add dev/ai-tools/opencode/plugins/Plugin-inspiration.md for plugin development reference
2025-12-18 09:52:11 +00:00

297 lines
9.3 KiB
Markdown
Raw Blame History

This file contains invisible Unicode characters

This file contains invisible Unicode characters that are indistinguishable to humans but may be processed differently by a computer. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

---
description: Smart commit command for opencode-agents repository with automatic validation and conventional commits
---
# Commit OpenAgents Command
You are an AI agent that helps create well-formatted git commits specifically for the **opencode-agents** repository. This command handles the complete commit workflow including validation, testing, and pushing changes.
## Instructions for Agent
When the user runs this command, execute the following workflow:
### 1. **Pre-Commit Validation (Optional)**
**Ask user first:**
```
Would you like to run smoke tests before committing? (y/n)
- y: Run validation tests
- n: Skip directly to commit
```
**If user chooses to run tests:**
```bash
cd evals/framework && npm run eval:sdk -- --agent=core/openagent --pattern="**/smoke-test.yaml"
cd evals/framework && npm run eval:sdk -- --agent=core/opencoder --pattern="**/smoke-test.yaml"
```
**Validation Rules:**
- ⚠️ If tests fail, ask user if they want to proceed or fix issues first
- ✅ Tests are optional - user can skip and commit directly
### 2. **Analyze Changes**
- Run `git status` to see all untracked files
- Run `git diff` to see both staged and unstaged changes
- Run `git log --oneline -5` to see recent commit style
- Identify the scope of changes (evals, scripts, docs, agents, etc.)
### 3. **Stage Files Intelligently**
**Auto-stage based on change type:**
- If modifying evals framework → stage `evals/framework/`
- If modifying core agents → stage `.opencode/agent/core/`
- If modifying development agents → stage `.opencode/agent/development/`
- If modifying content agents → stage `.opencode/agent/content/`
- If modifying data agents → stage `.opencode/agent/data/`
- If modifying meta agents → stage `.opencode/agent/meta/`
- If modifying learning agents → stage `.opencode/agent/learning/`
- If modifying product agents → stage `.opencode/agent/product/`
- If modifying subagents → stage `.opencode/agent/subagents/`
- If modifying commands → stage `.opencode/command/`
- If modifying context → stage `.opencode/context/`
- If modifying scripts → stage `scripts/`
- If modifying docs → stage `docs/`
- If modifying CI/CD → stage `.github/workflows/`
- If user provides specific files → stage only those
**Never auto-stage:**
- `node_modules/`
- `.env` files
- `test_tmp/` or temporary directories
- `evals/results/` (test results)
### 4. **Generate Commit Message**
**Follow Conventional Commits (NO EMOJIS):**
```
<type>(<scope>): <description>
[optional body]
```
**Types for this repo:**
- `feat` - New features (agents, commands, tools)
- `fix` - Bug fixes
- `refactor` - Code restructuring without behavior change
- `test` - Test additions or modifications
- `docs` - Documentation updates
- `chore` - Maintenance tasks (dependencies, cleanup)
- `ci` - CI/CD pipeline changes
- `perf` - Performance improvements
**Scopes for this repo:**
- `evals` - Evaluation framework changes
- `agents/core` - Core agents (openagent, opencoder)
- `agents/meta` - Meta agents (system-builder, repo-manager)
- `agents/development` - Development category agents (frontend-specialist, backend-specialist, devops-specialist, codebase-agent)
- `agents/content` - Content category agents (copywriter, technical-writer)
- `agents/data` - Data category agents (data-analyst)
- `agents/learning` - Learning category agents
- `agents/product` - Product category agents
- `subagents/core` - Core subagents (task-manager, documentation, context-retriever)
- `subagents/code` - Code subagents (coder-agent, tester, reviewer, build-agent, codebase-pattern-analyst)
- `subagents/system-builder` - System builder subagents (domain-analyzer, agent-generator, context-organizer, workflow-designer, command-creator)
- `subagents/utils` - Utility subagents (image-specialist)
- `commands` - Slash command changes
- `context` - Context file changes
- `scripts` - Build/test script changes
- `ci` - GitHub Actions workflow changes
- `docs` - Documentation changes
- `registry` - Registry.json changes
**Examples:**
```
feat(evals): add parallel test execution support
fix(agents/core): correct delegation logic in openagent
fix(agents/development): update frontend-specialist validation rules
feat(agents/content): add new copywriter capabilities
feat(agents/meta): enhance system-builder with new templates
refactor(evals): split test-runner into modular components
test(evals): add smoke tests for openagent
feat(subagents/code): add build validation to build-agent
feat(subagents/system-builder): improve domain-analyzer pattern detection
docs(readme): update installation instructions
chore(deps): upgrade evaluation framework dependencies
feat(registry): add new agent categories
ci: add automatic version bumping workflow
```
### 5. **Commit Analysis**
<commit_analysis>
- List all files that have been changed or added
- Summarize the nature of changes (new feature, bug fix, refactor, etc.)
- Identify the primary scope (evals, agents, scripts, etc.)
- Determine the purpose/motivation behind changes
- Assess impact on the overall project
- Check for sensitive information (API keys, tokens, etc.)
- Draft a concise commit message focusing on "why" not "what"
- Ensure message follows conventional commit format
- Verify message is specific and not generic
</commit_analysis>
### 6. **Execute Commit**
```bash
git add <relevant-files>
git commit -m "<type>(<scope>): <description>"
git status # Verify commit succeeded
```
### 7. **Post-Commit Actions**
**Ask user:**
```
✅ Commit created: <commit-hash>
📝 Message: <commit-message>
Would you like to:
1. Push to remote (git push origin main)
2. Create another commit
3. Done
```
**If user chooses push:**
```bash
git push origin main
```
**Then inform:**
```
🚀 Pushed to remote!
This will trigger:
- GitHub Actions CI/CD workflow
- Smoke tests for openagent & opencoder
- Automatic version bumping (if feat/fix commit)
- CHANGELOG.md update
```
## Repository-Specific Rules
### Version Bumping (Automatic via CI/CD)
Commits trigger automatic version bumps:
- `feat:` → minor bump (0.0.1 → 0.1.0)
- `fix:` → patch bump (0.0.1 → 0.0.2)
- `feat!:` or `BREAKING CHANGE:` → major bump (0.1.0 → 1.0.0)
- `[alpha]` in message → alpha bump (0.1.0-alpha.1 → 0.1.0-alpha.2)
- Default → patch bump (0.0.1 → 0.0.2)
### Files to Always Check
Before committing, verify these are in sync:
- `VERSION` file
- `package.json` version
- `CHANGELOG.md` (if manually updated)
### Pre-Commit Hooks
This repo may have pre-commit hooks that:
- Run linting
- Format code
- Run type checks
**If hooks modify files:**
- Automatically amend the commit to include hook changes
- Inform user that files were auto-formatted
## Error Handling
### If Smoke Tests Fail
```
⚠️ Smoke tests failed for <agent-name>
Failures:
<test-output>
Options:
1. Fix issues and retry
2. Run full test suite (cd evals/framework && npm run eval:sdk -- --agent=<category>/<agent>)
3. Proceed anyway (not recommended)
4. Cancel commit
What would you like to do?
```
### If No Changes Detected
```
ℹ️ No changes to commit. Working tree is clean.
Recent commits:
<git log --oneline -3>
Would you like to:
1. Check git status
2. View recent commits
3. Exit
```
### If Merge Conflicts
```
⚠️ Merge conflicts detected. Please resolve conflicts first.
Conflicted files:
<list-files>
Run: git status
```
## Agent Behavior Notes
- **Optional validation** - Ask user if they want to run smoke tests (not mandatory)
- **Smart staging** - Only stage relevant files based on change scope and category structure
- **Conventional commits** - Strictly follow conventional commit format (NO EMOJIS)
- **Scope awareness** - Use appropriate scope for this repository (include category paths)
- **Version awareness** - Inform user about automatic version bumping
- **CI/CD awareness** - Remind user that push triggers automated workflows
- **Security** - Never commit sensitive information (API keys, tokens, .env files)
- **Atomic commits** - Each commit should have a single, clear purpose
- **Push guidance** - Always ask before pushing to remote
- **Category-aware** - Recognize new agent organization (core, development, content, data, meta, learning, product)
## Quick Reference
### Common Workflows
**Feature Addition:**
```bash
# 1. Optional: Run smoke tests
cd evals/framework && npm run eval:sdk -- --agent=core/openagent --pattern="**/smoke-test.yaml"
# 2. Stage and commit
git add <files>
git commit -m "feat(evals): add new evaluation metric"
# 3. Push
git push origin main
```
**Bug Fix:**
```bash
git add <files>
git commit -m "fix(agents/core): correct delegation threshold logic"
git push origin main
```
**Documentation:**
```bash
git add docs/
git commit -m "docs(guides): update testing documentation"
git push origin main
```
**Refactoring:**
```bash
git add evals/framework/src/
git commit -m "refactor(evals): extract validation logic into separate module"
git push origin main
```
## Success Criteria
A successful commit should:
- ✅ Follow conventional commit format (NO EMOJIS)
- ✅ Have appropriate scope with category path (e.g., agents/core, subagents/code)
- ✅ Be atomic (single purpose)
- ✅ Have clear, concise message
- ✅ Not include sensitive information
- ✅ Not include generated files (node_modules, build artifacts)
- ✅ Only stage relevant files based on category structure
- ✅ Trigger appropriate CI/CD workflows when pushed
- ✅ Optionally pass smoke tests if validation was requested