Commit graph

13 commits

Author SHA1 Message Date
Darren Hinde
c8f7103cb6
refactor(evals): consolidate documentation and enhance test infrastructure (#56)
* feat(agents): implement category-based agent organization system

- Organize agents into domain categories (core, development, content, data, product, learning)
- Move 10 agents to category subdirectories with proper git rename tracking
- Update 13 subagents with category and type metadata in frontmatter
- Add category metadata files (0-category.json) documenting common patterns
- Implement local registry fallback in install script for offline development
- Add comprehensive validation suite with 15 automated tests (100% pass rate)
- Enhance registry validation with duplicate ID and consistency checks
- Update eval framework with intelligent path resolution (backward compatible)
- Archive legacy eval structure to _archive/ for reference
- Update all documentation to reflect category-based structure
- Bump version to 0.5.0 with accurate CHANGELOG

Technical Details:
- 23 agents organized (10 category agents, 13 subagents)
- 6 category directories created
- Path resolution supports both agent IDs and category paths
- Registry schema updated to v2.0.0
- 159 files changed, 872 insertions(+), 10738 deletions(-)

BREAKING CHANGE: Agent file paths now use category structure. Update references from .opencode/agent/openagent.md to .opencode/agent/core/openagent.md. Eval framework maintains backward compatibility via path resolution.

* refactor(evals): consolidate documentation and enhance test infrastructure

- Remove temporary project tracking files (PHASE_5_COMPLETE, PROJECT_COMPLETE, etc.)
- Consolidate evaluation framework docs into main README
- Enhance test execution with improved logging and multi-prompt support
- Move system-builder from core to meta category
- Add comprehensive test suites for openagent with organized structure
- Create evaluation test structure for all subagents
- Clean up archived workflows and redundant documentation
- Update registry to reflect new agent organization
- Add shared test templates and golden test patterns

* feat(ci): add manual trigger support for bot-created PRs in validate-registry workflow

- Add pr_number input to workflow_dispatch for manually triggering validation
- Fetch PR details dynamically when triggered manually
- Support both automatic (PR event) and manual (workflow_dispatch) triggers
- Enable validation of bot-created PRs like automated version bumps
- Update branch detection and push logic to handle both trigger types
- Add documentation explaining how to manually trigger for bot PRs

* feat(evals): add explicit context file validation to test framework

- Add expectedContextFiles field to test YAML schema for explicit context file specification
- Enhance context-loading-evaluator to support both auto-detect and explicit validation modes
- Update documentation with comprehensive guide and examples
- Clean up archived legacy test structure (90+ old test files)
- Add new example tests demonstrating explicit context validation
- Backward compatible with existing tests

* feat(evals): add multi-agent logging system and performance optimizations

Implement comprehensive multi-agent logging and performance improvements for the eval framework.

Task 01: Multi-Agent Logging System
- Add complete hierarchical logging module (evals/framework/src/logging/)
  - SessionTracker: tracks parent-child delegation hierarchies
  - MultiAgentLogger: pretty-prints logs with visual indentation
  - Formatters: box characters and emoji formatting
  - 37 passing unit tests (session-tracker, logger, integration)
- Integrate with SDK event stream handler
  - Hook into session.created, message.updated, message.part.updated events
  - Real-time child session detection via timestamp heuristics
  - Message deduplication for cleaner output
- Enable in debug mode only (<1% performance overhead)
- Add demo script and comprehensive documentation

Task 02: Performance Optimizations
- Reduce grace period from 5s to 2s (67% reduction, 10-20% faster tests)
- Add PerformanceMetricsEvaluator for bottleneck identification
  - Collects tool latencies, inference time, idle time
  - Provides full performance visibility
- Update smoke test to validate delegation (core feature)
  - Replace simple read test with multi-agent delegation test
  - Tests both parent and child agent functionality
  - Validates multi-agent logging system

Results:
- 37 unit tests passing
- Smoke test passing (95/100 score)
- Clean hierarchical logging output
- 10-20% faster test execution
- Full multi-agent visibility

Files changed: 16 files, +2678 lines
Time saved: ~6-8 days (completed in 6 hours vs 3-5 day estimate)

* feat(evals): show child agent execution in non-debug mode

- Enable MultiAgentLogger in both debug and non-debug modes
- Add verbose parameter to control output level
- Non-verbose mode shows concise child session lifecycle:
  - Child agent started message
  - Child agent completed with duration
- Verbose mode (--debug) shows full delegation hierarchy
- Update documentation to reflect new behavior

This provides visibility into delegation without overwhelming output,
giving confidence that child agents are actually running.

* feat(evals): improve delegation testing and behavior validation

Major improvements to eval framework:

Test Suite:
- Add 00-smoke-test.yaml (basic read operation)
- Rename 01-smoke-test.yaml → 02-delegation-test.yaml (delegation test)
- Add simple-responder test agent for delegation testing
- Add debug test: simple-subagent-call.yaml

Evaluators:
- Add agent-model-evaluator.ts for agent/model validation
- Enhance behavior-evaluator.ts with detailed task tool output
- Improve delegation-evaluator.ts with better evidence tracking

Schema & Execution:
- Add expectedAgent and expectedModel to test schema
- Improve test executor with better logging and error handling

Documentation:
- Update evals/README.md with new features and performance improvements
- Remove outdated MULTI_AGENT_LOGGING_COMPLETE.md

Registry:
- Add simple-responder test agent to registry

Fixes:
- Fix dashboard.sh path resolution

These changes provide better visibility into delegation, improved test
coverage, and clearer validation of agent behavior.

* chore: add GitHub Actions workflow and plugin documentation

- Add .github/workflows/evals/run-evaluations.yml for automated eval testing
- Add dev/ai-tools/opencode/plugins/Plugin-inspiration.md for plugin development reference
2025-12-18 09:52:11 +00:00
darrenhinde
c6775bbc40 chore: consolidate development changes and refactor agent system
Major Changes:
- Rename codebase-agent to opencoder for clarity
- Add agent validator plugin with comprehensive testing
- Refactor prompt engineering commands (enhancer → optimizer)
- Update task-manager subagent with improved workflows
- Enhance plugin system with validation and documentation

Agent System:
- Renamed: codebase-agent.md → opencoder.md
- Updated: task-manager.md with better delegation patterns
- Added: opencoder.md documentation

Plugin System:
- New: agent-validator.ts for prompt validation
- New: VALIDATOR_GUIDE.md with usage instructions
- New: Test suite (test-validator.sh, test-validation.sh)
- Updated: notify.ts, telegram-notify.ts with improvements
- Updated: package.json, bun.lock with new dependencies

Commands:
- Renamed: prompt-enchancer.md → prompt-engineering/prompt-enhancer.md
- New: prompt-engineering/prompt-optimizer.md
- Updated: build-context-system.md, validate-repo.md

Documentation:
- Updated: README.md with latest features
- Updated: openagent.md documentation
- New: opencoder.md agent documentation
- Updated: agent-system-blueprint.md
- Updated: system-builder docs (README, quick-start)
- Updated: installation.md, collision-handling.md

Registry:
- Updated: registry.json with new components

This consolidates all development work for the upcoming release.
2025-11-21 13:30:18 +00:00
darrenhinde
8b75f128d3 docs: update to recommend OpenAgent as default with enhanced workflow diagrams
- Position OpenAgent as recommended default for new users
- Add detailed stage diagrams for all 6 workflow stages
- Replace bright purple colors with softer slate blue for better accessibility
- Update all documentation to show OpenAgent as universal coordinator
- Keep codebase-agent as specialized development alternative
2025-11-18 23:15:44 +00:00
darrenhinde
ca323285b2 Rename to OpenAgents, remove model specs, and update documentation
Major Changes:
- Rename repository from opencode-agents to OpenAgents
- Remove all hardcoded model specifications (now model-agnostic)
- Fix documentation: profile names, component counts, add Business profile
- Add future plans note for other AI coding tools

Repository Rename:
- Update all URLs to github.com/darrenhinde/OpenAgents
- Update git remote, README, install scripts, documentation
- 15 files updated with new repository name

Model-Agnostic Framework:
- Remove model specifications from 6 agent files
- Framework now works with any AI provider (Claude, GPT, Gemini, local models)
- Users control model selection via OpenCode CLI settings

Documentation Fixes:
- Correct profile name: Core → Essential
- Add missing Business profile documentation
- Fix component counts: Developer (19), Advanced (32)
- Document all 5 profiles with accurate descriptions

Future Plans:
- Add note about support for other AI tools (Cursor, Claude Code, etc.)
- Clear focus on stabilizing OpenCode integration first

This makes OpenAgents a truly universal, model-agnostic framework
accessible to all users regardless of their AI provider preference.
2025-11-18 22:37:41 +00:00
darrenhinde
5f2cec1404 ✨ feat: introduce OpenAgent universal agent with session management
Major architectural refactoring:

- Add OpenAgent as universal primary agent for questions, tasks, and workflows
- Implement session-based context management with manifest tracking
- Reorganize subagents into logical categories (code/core/utils)
- Add dynamic context discovery and loading system
- Introduce cleanup script for stale sessions
- Update registry with new categorization (essential/standard/extended/specialized/meta)
- Rename 'core' profile to 'essential' and add 'business' profile
- Add comprehensive documentation with visual diagrams
- Update installation script with new profiles

Breaking changes:
- Subagent paths changed from analysis/core to code/core/utils
- Profile 'core' renamed to 'essential' (backward compatible)

New features:
- Session isolation for concurrent agent instances (.tmp/sessions/)
- Automatic stale session cleanup (>24 hours)
- Manifest-based context indexing and discovery
- Business profile for content-focused workflows
- OpenAgent with plan-approve-execute-validate-summarize workflow

Components: 32 total (3 agents, 13 subagents, 8 commands, 2 tools, 2 plugins, 2 contexts, 2 config)

Documentation:
- docs/agents/openagent.md - Comprehensive guide with Mermaid diagrams
- docs/guides/building-with-opencode.md - Power user formula
- scripts/README.md - Script documentation
- .opencode/context/core/context-management.md - Session management strategy
2025-11-18 18:45:42 +00:00
darrenhinde
e524ba6ded feat: add system builder and reorganize project structure
New Features:
- Add system-builder agent for generating custom AI systems
- Add 5 new subagents for system building:
  - domain-analyzer: Analyzes domains and recommends architectures
  - agent-generator: Generates XML-optimized agent files
  - context-organizer: Organizes modular context files
  - workflow-designer: Designs workflow definitions
  - command-creator: Creates custom slash commands
- Add build-context-system command for interactive system generation
- Add system builder templates and guide

Project Reorganization:
- Move GITHUB_SETTINGS.md to docs/github/
- Move AGENT-SYSTEM-BLUEPRINT.md to docs/features/
- Move CODE_OF_CONDUCT.md to docs/contributing/
- Move CONTRIBUTING.md to docs/contributing/
- Consolidate documentation in docs/ directory

Registry Updates:
- Add system-builder agent to registry
- Add 5 new subagents to registry
- Add build-context-system command to registry
- Update advanced profile to include system builder components

This enables users to generate complete custom AI systems tailored to their domain through an interactive interview process.
2025-11-18 14:00:35 +00:00
darrenhinde
80e1a8d650 ✨ feat: make agent system language-agnostic with multi-language support
- Make codebase-agent work with TypeScript, Python, Go, Rust, and more
- Add clear runtime/type-check/lint command examples for each language
- Update build-agent to support all languages (tsc, mypy, go build, cargo check)
- Make essential-patterns universal by removing language-specific code examples
- Update README to mention multi-language support
- Remove auto-detection overhead - agents naturally adapt based on project files
2025-11-17 00:16:18 +00:00
darrenhinde
cac7ee6b03 📝 docs: update codebase-agent.md to include @image-specialist subagent and remove GEMINI_API_KEY requirement from image-specialist.md 2025-09-24 15:35:32 +01:00
darrenhinde
e073c5bb2e 🗑️ chore: remove deprecated Gemini tool files and update package configuration for improved structure 2025-09-23 13:31:36 +01:00
darrenhinde
b4927fc1b0 🔧 chore: update agent documentation and enhance prompt architecture with structured XML methodology 2025-09-18 16:02:28 +01:00
darrenhinde
a585e72439 🔧 chore: update agent models and enhance command specifications 2025-08-29 17:51:53 +01:00
darrenhinde
a494e84a39 Remove unused agent and task manager files to streamline the codebase and improve maintainability. 2025-08-16 00:00:57 +01:00
darrenhinde
0119898081 Update README.md to enhance project description, add features, quick start guide, agent and plugin details, configuration instructions, and safety/security practices. 2025-08-15 01:43:56 +01:00