bf02ad0d5c
* refactor: Rename repository to OpenAgents Control Update all project references from "OpenAgents" to "OpenAgents Control": - Updated core config files (package.json, registry.json) - Updated documentation (README.md, docs/) - Updated scripts (install.sh, update.sh, test scripts) - Updated agent and command files - Updated integrations and packages - Updated all GitHub URLs to use OpenAgentsControl Naming convention: - Display name: "OpenAgents Control" - Repository URL: OpenAgentsControl * feat: Enable subagent-to-subagent delegation (ContextScout integration) Allow coding-related subagents to call ContextScout for context discovery: - CoderAgent: Can now discover coding standards on-demand - TestEngineer: Can now discover test coverage and TDD guidelines - CodeReviewer: Can now discover code review and security patterns - BuildAgent: Can now discover build validation standards - DocWriter: Can now discover documentation formatting guidelines Changes per subagent: - Added `task: true` to tools section - Added permission allowlist for ContextScout only (deny all other delegation) - Added Context Discovery workflow section with when-to-call guidance This enables subagents to be self-sufficient for finding relevant standards instead of relying on orchestrator to provide all context files. Benefits: - Subagents can discover what they need when context isn't explicit - Maintains security (only ContextScout allowed via task tool) - Follows existing patterns (uses task tool like main agents) * fix: Set OPENCODE_BRANCH in CI tests to use PR branch for downloads - Add OPENCODE_BRANCH environment variable to E2E tests - Add OPENCODE_BRANCH environment variable to profile smoke tests - Ensures tests download files from the PR branch instead of main - Fixes E2E test failures after repository rename to OpenAgentsControl
86 lines
2.2 KiB
Markdown
86 lines
2.2 KiB
Markdown
# Repository Manager Agent - Eval Tests
|
|
|
|
**Agent**: `meta/repo-manager`
|
|
**Version**: 1.0.0
|
|
**Category**: Meta
|
|
**Type**: Meta Agent
|
|
|
|
---
|
|
|
|
## Overview
|
|
|
|
Eval tests for the Repository Manager meta agent. This agent orchestrates OpenAgents Control repository development with context-aware planning, task breakdown, and automatic documentation.
|
|
|
|
## Test Suites
|
|
|
|
### Smoke Test (`smoke-test.yaml`)
|
|
Basic functionality verification:
|
|
- Context loading for informational queries
|
|
- Approval gate enforcement
|
|
- Context awareness for repo-specific questions
|
|
|
|
**Run**:
|
|
```bash
|
|
cd evals/framework
|
|
npm run eval:sdk -- --agent=meta/repo-manager --pattern="smoke-test.yaml"
|
|
```
|
|
|
|
### Context Loading Test (`context-loading-test.yaml`)
|
|
Verifies appropriate context loading for different task types:
|
|
- Agent creation tasks
|
|
- Eval testing tasks
|
|
- Registry management tasks
|
|
|
|
**Run**:
|
|
```bash
|
|
cd evals/framework
|
|
npm run eval:sdk -- --agent=meta/repo-manager --pattern="context-loading-test.yaml"
|
|
```
|
|
|
|
### Delegation Test (`delegation-test.yaml`)
|
|
Verifies delegation decisions and context bundle creation:
|
|
- Complex features → delegate to task-manager
|
|
- Simple tasks → execute directly
|
|
- Documentation → delegate to documentation subagent
|
|
|
|
**Run**:
|
|
```bash
|
|
cd evals/framework
|
|
npm run eval:sdk -- --agent=meta/repo-manager --pattern="delegation-test.yaml"
|
|
```
|
|
|
|
## Running All Tests
|
|
|
|
```bash
|
|
cd evals/framework
|
|
npm run eval:sdk -- --agent=meta/repo-manager
|
|
```
|
|
|
|
## Test Coverage
|
|
|
|
**Context Loading**: ✅ Covered
|
|
**Approval Gates**: ✅ Covered
|
|
**Delegation Logic**: ✅ Covered
|
|
**Context Bundles**: ✅ Covered
|
|
**Validation**: ⚠️ Partial (needs integration tests)
|
|
**Documentation Updates**: ⚠️ Partial (needs integration tests)
|
|
|
|
## Known Limitations
|
|
|
|
- Integration tests for full workflow not yet implemented
|
|
- Validation script execution not tested in isolation
|
|
- Documentation update detection needs more test cases
|
|
|
|
## Future Test Cases
|
|
|
|
- [ ] Full agent creation workflow (end-to-end)
|
|
- [ ] Validation failure handling
|
|
- [ ] Documentation update detection
|
|
- [ ] Session cleanup verification
|
|
- [ ] Multi-step task coordination
|
|
- [ ] Error recovery scenarios
|
|
|
|
---
|
|
|
|
**Last Updated**: 2025-01-21
|
|
**Maintainer**: opencode
|