OpenWriterControle/docs/agents/repo-manager.md
Darren Hinde c8f7103cb6
refactor(evals): consolidate documentation and enhance test infrastructure (#56)
* feat(agents): implement category-based agent organization system

- Organize agents into domain categories (core, development, content, data, product, learning)
- Move 10 agents to category subdirectories with proper git rename tracking
- Update 13 subagents with category and type metadata in frontmatter
- Add category metadata files (0-category.json) documenting common patterns
- Implement local registry fallback in install script for offline development
- Add comprehensive validation suite with 15 automated tests (100% pass rate)
- Enhance registry validation with duplicate ID and consistency checks
- Update eval framework with intelligent path resolution (backward compatible)
- Archive legacy eval structure to _archive/ for reference
- Update all documentation to reflect category-based structure
- Bump version to 0.5.0 with accurate CHANGELOG

Technical Details:
- 23 agents organized (10 category agents, 13 subagents)
- 6 category directories created
- Path resolution supports both agent IDs and category paths
- Registry schema updated to v2.0.0
- 159 files changed, 872 insertions(+), 10738 deletions(-)

BREAKING CHANGE: Agent file paths now use category structure. Update references from .opencode/agent/openagent.md to .opencode/agent/core/openagent.md. Eval framework maintains backward compatibility via path resolution.

* refactor(evals): consolidate documentation and enhance test infrastructure

- Remove temporary project tracking files (PHASE_5_COMPLETE, PROJECT_COMPLETE, etc.)
- Consolidate evaluation framework docs into main README
- Enhance test execution with improved logging and multi-prompt support
- Move system-builder from core to meta category
- Add comprehensive test suites for openagent with organized structure
- Create evaluation test structure for all subagents
- Clean up archived workflows and redundant documentation
- Update registry to reflect new agent organization
- Add shared test templates and golden test patterns

* feat(ci): add manual trigger support for bot-created PRs in validate-registry workflow

- Add pr_number input to workflow_dispatch for manually triggering validation
- Fetch PR details dynamically when triggered manually
- Support both automatic (PR event) and manual (workflow_dispatch) triggers
- Enable validation of bot-created PRs like automated version bumps
- Update branch detection and push logic to handle both trigger types
- Add documentation explaining how to manually trigger for bot PRs

* feat(evals): add explicit context file validation to test framework

- Add expectedContextFiles field to test YAML schema for explicit context file specification
- Enhance context-loading-evaluator to support both auto-detect and explicit validation modes
- Update documentation with comprehensive guide and examples
- Clean up archived legacy test structure (90+ old test files)
- Add new example tests demonstrating explicit context validation
- Backward compatible with existing tests

* feat(evals): add multi-agent logging system and performance optimizations

Implement comprehensive multi-agent logging and performance improvements for the eval framework.

Task 01: Multi-Agent Logging System
- Add complete hierarchical logging module (evals/framework/src/logging/)
  - SessionTracker: tracks parent-child delegation hierarchies
  - MultiAgentLogger: pretty-prints logs with visual indentation
  - Formatters: box characters and emoji formatting
  - 37 passing unit tests (session-tracker, logger, integration)
- Integrate with SDK event stream handler
  - Hook into session.created, message.updated, message.part.updated events
  - Real-time child session detection via timestamp heuristics
  - Message deduplication for cleaner output
- Enable in debug mode only (<1% performance overhead)
- Add demo script and comprehensive documentation

Task 02: Performance Optimizations
- Reduce grace period from 5s to 2s (67% reduction, 10-20% faster tests)
- Add PerformanceMetricsEvaluator for bottleneck identification
  - Collects tool latencies, inference time, idle time
  - Provides full performance visibility
- Update smoke test to validate delegation (core feature)
  - Replace simple read test with multi-agent delegation test
  - Tests both parent and child agent functionality
  - Validates multi-agent logging system

Results:
- 37 unit tests passing
- Smoke test passing (95/100 score)
- Clean hierarchical logging output
- 10-20% faster test execution
- Full multi-agent visibility

Files changed: 16 files, +2678 lines
Time saved: ~6-8 days (completed in 6 hours vs 3-5 day estimate)

* feat(evals): show child agent execution in non-debug mode

- Enable MultiAgentLogger in both debug and non-debug modes
- Add verbose parameter to control output level
- Non-verbose mode shows concise child session lifecycle:
  - Child agent started message
  - Child agent completed with duration
- Verbose mode (--debug) shows full delegation hierarchy
- Update documentation to reflect new behavior

This provides visibility into delegation without overwhelming output,
giving confidence that child agents are actually running.

* feat(evals): improve delegation testing and behavior validation

Major improvements to eval framework:

Test Suite:
- Add 00-smoke-test.yaml (basic read operation)
- Rename 01-smoke-test.yaml → 02-delegation-test.yaml (delegation test)
- Add simple-responder test agent for delegation testing
- Add debug test: simple-subagent-call.yaml

Evaluators:
- Add agent-model-evaluator.ts for agent/model validation
- Enhance behavior-evaluator.ts with detailed task tool output
- Improve delegation-evaluator.ts with better evidence tracking

Schema & Execution:
- Add expectedAgent and expectedModel to test schema
- Improve test executor with better logging and error handling

Documentation:
- Update evals/README.md with new features and performance improvements
- Remove outdated MULTI_AGENT_LOGGING_COMPLETE.md

Registry:
- Add simple-responder test agent to registry

Fixes:
- Fix dashboard.sh path resolution

These changes provide better visibility into delegation, improved test
coverage, and clearer validation of agent behavior.

* chore: add GitHub Actions workflow and plugin documentation

- Add .github/workflows/evals/run-evaluations.yml for automated eval testing
- Add dev/ai-tools/opencode/plugins/Plugin-inspiration.md for plugin development reference
2025-12-18 09:52:11 +00:00

497 lines
17 KiB
Markdown

# Repository Manager Agent v2.0
**Agent ID**: `repo-manager`
**Category**: Meta
**Type**: Meta Agent
**Version**: 2.0.0
**Status**: Stable
---
## Overview
The Repository Manager is a meta agent specifically designed for managing development work on the OpenAgents repository itself. It provides **lazy context loading**, **smart delegation**, and **automatic documentation updates** with intelligent subagent coordination.
### What's New in v2.0
- ✅ **Lazy Context Loading** - Uses `context-retriever` for dynamic context discovery
- ✅ **Smart Delegation** - Session files for complex tasks, inline context for simple
- ✅ **Predictable Workflow** - Same 6-stage process every time
- ✅ **No Hardcoded Paths** - Adapts to any repo structure
- ✅ **Clear Decision Trees** - Explicit logic for when to delegate vs execute
---
## Purpose
While `openagent` is a universal agent for any project, `repo-manager` is specialized for OpenAgents repository development. It understands the repository's structure, conventions, and standards, ensuring all work follows established patterns.
---
## Key Features
### 1. **Lazy Context Loading**
- Uses `context-retriever` subagent for dynamic discovery
- Loads context **just-in-time** (after approval, before execution)
- No hardcoded paths - adapts to repo changes
- Only loads what's needed for the specific task
### 2. **Smart Delegation Strategy**
- **Complex tasks** (4+ files, >60min) → Session file + task-manager
- **Simple specialists** (tester, reviewer) → Inline context
- **Direct execution** (1-3 files) → No delegation
### 3. **Session-Based Coordination**
- Creates `.tmp/sessions/{timestamp}-{task-slug}/` for complex tasks
- Context file contains: user request, context files to load, requirements, exit criteria
- Shared memory for subagent coordination
- Automatic cleanup after completion
### 4. **Approval-Gated Workflow**
- Never executes without explicit user approval
- Presents clear implementation plans before proceeding
- Stops on validation failures and requests approval for fixes
### 5. **Automatic Documentation Updates**
- Identifies documentation affected by changes
- Updates docs automatically or delegates to documentation subagent
- Keeps README files, guides, and context files current
### 6. **Validation Integration**
- Runs validation scripts automatically (registry, test suites)
- Executes eval tests when applicable
- Stops on failure and reports issues clearly
---
## When to Use
Use `repo-manager` for:
- ✅ Creating new agents
- ✅ Creating/modifying eval tests
- ✅ Updating the registry
- ✅ Managing context files
- ✅ Updating documentation
- ✅ Any OpenAgents repository development
Don't use `repo-manager` for:
- ❌ Working on user projects (use `openagent` instead)
- ❌ General coding tasks outside this repo
- ❌ Questions about other projects
---
## The 6-Stage Workflow
Every task follows the same predictable workflow:
```
┌─────────────────────────────────────────────────────────────┐
│ STAGE 1: ANALYZE │
│ - Classify task type (agent, eval, registry, docs, etc.) │
│ - Determine complexity (simple vs complex) │
│ - Decide execution path (question vs task) │
└─────────────────────────────────────────────────────────────┘
↓
┌─────────────────────────────────────────────────────────────┐
│ STAGE 2: PLAN & APPROVE │
│ - Create implementation plan │
│ - List files to create/modify │
│ - Identify context needed (don't load yet) │
│ - Request approval │
│ - ✅ Wait for user approval │
└─────────────────────────────────────────────────────────────┘
↓
┌─────────────────────────────────────────────────────────────┐
│ STAGE 3: LOAD CONTEXT (Lazy Loading) │
│ - Load quick-start.md (always) │
│ - Delegate to context-retriever for discovery │
│ - Load discovered context files │
│ - Extract key requirements │
└─────────────────────────────────────────────────────────────┘
↓
┌─────────────────────────────────────────────────────────────┐
│ STAGE 4: EXECUTE │
│ - Decision: Complex → Session file + delegate │
│ - Decision: Simple specialist → Inline context │
│ - Decision: Direct → Execute with loaded context │
└─────────────────────────────────────────────────────────────┘
↓
┌─────────────────────────────────────────────────────────────┐
│ STAGE 5: VALIDATE │
│ - Run validation scripts │
│ - Run tests if applicable │
│ - On failure: STOP → REPORT → PROPOSE → APPROVE → FIX │
└─────────────────────────────────────────────────────────────┘
↓
┌─────────────────────────────────────────────────────────────┐
│ STAGE 6: COMPLETE │
│ - Update affected documentation │
│ - Summarize all changes │
│ - Confirm user satisfaction │
│ - Cleanup session files (if created) │
└─────────────────────────────────────────────────────────────┘
```
---
## Lazy Context Loading (Stage 3)
Instead of hardcoding context file paths, repo-manager uses **context-retriever** for dynamic discovery:
### How It Works
1. **Load quick-start.md** (always - orientation file)
```
Read: .opencode/context/openagents-repo/quick-start.md
```
2. **Delegate to context-retriever** for discovery
```javascript
task(
subagent_type="subagents/core/context-retriever",
description="Find context for {task-type}",
prompt="Search for context files related to: {task-type}
Return:
- Exact file paths
- Brief summaries
- Priority order"
)
```
3. **Load discovered files** in priority order
```
FOR EACH file in discovered_files:
Read: {file-path}
```
4. **Extract requirements** from loaded context
- Naming conventions
- File structure requirements
- Validation requirements
- Testing requirements
### Benefits
- ✅ No hardcoded paths - adapts to repo changes
- ✅ Only loads what's needed - true lazy loading
- ✅ Discovers new context files automatically
- ✅ Works even if context structure changes
---
## Smart Delegation (Stage 4)
Repo-manager uses three delegation strategies based on task complexity:
### Strategy 1: Session File Delegation (Complex Tasks)
**When**: 4+ files, >60min, complex dependencies, task breakdown needed
**Subagents**: task-manager, documentation
**Process**:
1. Create session directory: `.tmp/sessions/{timestamp}-{task-slug}/`
2. Write `context.md` with:
- User request
- Context files to load
- Key requirements (extracted from Stage 3)
- Files to create/modify
- Exit criteria
3. Delegate with context path
4. Cleanup after completion
**Example**:
```javascript
task(
subagent_type="subagents/core/task-manager",
prompt="Load context from .tmp/sessions/20250114-143022-parallel-tests/context.md
Break down this feature into atomic subtasks.
Follow all requirements in context file."
)
```
### Strategy 2: Inline Context Delegation (Simple Specialists)
**When**: Simple delegation to specialists (tester, reviewer, coder-agent)
**Subagents**: tester, reviewer, coder-agent, build-agent
**Process**:
1. No session file needed
2. Pass context directly in prompt
3. Include extracted requirements from Stage 3
**Example**:
```javascript
task(
subagent_type="subagents/code/tester",
prompt="Context to load:
- .opencode/context/core/standards/tests.md
Task: Write tests for {feature}
Requirements:
- Positive and negative test cases
- Arrange-Act-Assert pattern
Files to test:
- {file1}
- {file2}"
)
```
### Strategy 3: Direct Execution (No Delegation)
**When**: 1-3 files, straightforward, <30min
**Process**:
1. Execute directly using context loaded in Stage 3
2. Apply requirements extracted from context
3. Create/modify files as planned
---
## Available Subagents
### Core Subagents (Planning & Coordination)
| Subagent | Purpose | Context Strategy |
|----------|---------|------------------|
| **task-manager** | Break down complex features into atomic subtasks | Session file |
| **context-retriever** | Find and retrieve relevant context files | None (discovery) |
| **documentation** | Generate/update comprehensive documentation | Session file |
### Code Subagents (Implementation & Quality)
| Subagent | Purpose | Context Strategy |
|----------|---------|------------------|
| **coder-agent** | Execute simple coding subtasks | Inline |
| **tester** | Write tests following TDD | Inline |
| **reviewer** | Code review, security, quality checks | Inline |
| **build-agent** | Type checking, build validation | Inline |
---
## Session File Structure
When creating session files for complex tasks:
```
.tmp/sessions/{timestamp}-{task-slug}/
├── context.md # Shared context for all subagents
└── .manifest.json # Track session metadata
```
### context.md Template
```markdown
# Task Context: {Task Name}
Session ID: {timestamp}-{task-slug}
Created: {ISO timestamp}
Status: in_progress
## Current Request
{Original user request}
## Context Files to Load
- .opencode/context/openagents-repo/quick-start.md
- {other context files discovered in Stage 3}
## Key Requirements (Extracted from Context)
- {requirement 1}
- {requirement 2}
## Files to Create/Modify
- {file 1} - {purpose}
- {file 2} - {purpose}
## Technical Constraints
{Any technical constraints}
## Exit Criteria
- [ ] {criteria 1}
- [ ] {criteria 2}
## Progress Tracking
- [ ] Context loaded and understood
- [ ] Implementation complete
- [ ] Tests passing
- [ ] Documentation updated
---
**Instructions for Subagent**:
{Specific instructions}
```
---
## Examples
### Example 1: Simple Agent Creation (Direct Execution)
**User**: "Create a new data analyst agent"
**Workflow**:
1. **Analyze**: Task type = agent-creation, Complexity = simple (4 files)
2. **Plan**: Present plan to create 4 files, request approval
3. **LoadContext**:
- Load quick-start.md
- Delegate to context-retriever: "Find context for agent-creation"
- Load discovered files: agents.md, adding-agent.md, code.md
- Extract: frontmatter format, category structure, naming conventions
4. **Execute**: Direct execution (4 files, straightforward)
- Create agent file with proper frontmatter
- Create eval structure
- Update registry
5. **Validate**: Run registry validation, smoke test
6. **Complete**: Update docs, summarize, confirm
**Context Flow**: ✅ Lazy loaded, ✅ No session files, ✅ Direct execution
---
### Example 2: Complex Feature (Delegation Chain)
**User**: "Build parallel test execution for eval framework"
**Workflow**:
1. **Analyze**: Task type = general-development, Complexity = complex (6+ files, >60min)
2. **Plan**: Present plan to delegate to task-manager, request approval
3. **LoadContext**:
- Load quick-start.md
- Delegate to context-retriever: "Find context for eval framework development"
- Load discovered files: evals.md, code.md, tests.md, patterns.md
- Extract: modular patterns, test requirements, eval structure
4. **Execute**: Session delegation
- Create `.tmp/sessions/20250114-143022-parallel-tests/context.md`
- Delegate to task-manager with context path
- Task-manager creates subtasks
- Implement subtasks (delegate to coder-agent or execute directly)
5. **Validate**:
- Run tests
- Delegate to tester for validation
- Delegate to reviewer for quality check
6. **Complete**:
- Delegate to documentation for docs update
- Summarize changes
- Cleanup session files
**Context Flow**: ✅ Lazy loaded, ✅ Session file created, ✅ Shared memory, ✅ Clean coordination
---
## Decision Matrix
| Scenario | Context Loading | Session File? | Delegation |
|----------|----------------|---------------|------------|
| Simple agent creation | Lazy (context-retriever) | ❌ No | None (direct) |
| Complex feature (4+ files) | Lazy (context-retriever) | ✅ Yes | task-manager |
| Write tests | Lazy (context-retriever) | ❌ No | tester (inline) |
| Code review | Lazy (context-retriever) | ❌ No | reviewer (inline) |
| Simple implementation | Lazy (context-retriever) | ❌ No | coder-agent (inline) |
| Comprehensive docs | Lazy (context-retriever) | ✅ Yes | documentation |
| Find context files | None (discovery agent) | ❌ No | context-retriever |
---
## Quality Standards
### Repository Conventions
- Follow category-based organization (core, development, content, etc.)
- Use proper frontmatter metadata in agent files
- Follow naming conventions (kebab-case for files)
- Include proper tags and dependencies
- Validate against registry schema
### Context Awareness
- Load quick-start.md for every task
- Use context-retriever for dynamic discovery
- Apply standards from core/standards/
- Follow workflows from core/workflows/
- Reference context files (don't duplicate content)
### Documentation Maintenance
- Update docs when making changes
- Keep README files current
- Update version/date stamps
- Maintain consistency across docs
- Follow docs standards
### Validation Requirements
- Run validation scripts before completion
- Test agents with eval framework
- Validate registry entries
- Check file structure
- Verify documentation links
---
## Critical Rules
1. **Approval Gate**: Request approval before ANY execution (bash, write, edit, task)
2. **Context First**: Load repo context BEFORE executing (lazy, via context-retriever)
3. **Stop on Failure**: STOP on test fail/errors - NEVER auto-fix
4. **Report First**: On fail: REPORT→PROPOSE FIX→REQUEST APPROVAL→FIX
5. **Confirm Cleanup**: Confirm before deleting session files
---
## Testing
**Eval Tests**: `evals/agents/meta/repo-manager/`
**Test Suites**:
- `smoke-test.yaml` - Basic functionality
- `context-loading-test.yaml` - Lazy context loading
- `delegation-test.yaml` - Delegation decisions and session files
**Run Tests**:
```bash
cd evals/framework
npm run eval:sdk -- --agent=meta/repo-manager
```
**Run Specific Test**:
```bash
cd evals/framework
npm run eval:sdk -- --agent=meta/repo-manager --pattern="smoke-test.yaml"
```
---
## Comparison: v1.0 vs v2.0
| Aspect | v1.0 | v2.0 |
|--------|------|------|
| **Context Loading** | Upfront (hardcoded paths) | Lazy (context-retriever) |
| **Session Files** | Always creates bundles | Only for complex tasks |
| **Context Passing** | Always via files | Smart (inline vs session) |
| **Workflow Stages** | 8 stages | 6 stages |
| **Lines of Code** | 763 lines | ~650 lines |
| **Adaptability** | Hardcoded paths | Dynamic discovery |
| **Clarity** | Over-specified | Clear decision trees |
---
## Principles
1. **Lazy Loading**: Fetch context when needed via context-retriever, not before
2. **Smart Delegation**: Session files for complex, inline for simple
3. **Safety First**: Always request approval, stop on failure
4. **Quality Focused**: Validate against repo standards, never auto-fix
5. **Adaptive**: Direct execution for simple, delegation for complex
6. **Discoverable**: Use context-retriever for dynamic context discovery
7. **Predictable**: Same workflow every time
---
**Last Updated**: 2025-01-14
**Maintainer**: opencode
**Status**: Stable
**Version**: 2.0.0