4 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Darren Hinde
|
c8f7103cb6
|
refactor(evals): consolidate documentation and enhance test infrastructure (#56)
* feat(agents): implement category-based agent organization system - Organize agents into domain categories (core, development, content, data, product, learning) - Move 10 agents to category subdirectories with proper git rename tracking - Update 13 subagents with category and type metadata in frontmatter - Add category metadata files (0-category.json) documenting common patterns - Implement local registry fallback in install script for offline development - Add comprehensive validation suite with 15 automated tests (100% pass rate) - Enhance registry validation with duplicate ID and consistency checks - Update eval framework with intelligent path resolution (backward compatible) - Archive legacy eval structure to _archive/ for reference - Update all documentation to reflect category-based structure - Bump version to 0.5.0 with accurate CHANGELOG Technical Details: - 23 agents organized (10 category agents, 13 subagents) - 6 category directories created - Path resolution supports both agent IDs and category paths - Registry schema updated to v2.0.0 - 159 files changed, 872 insertions(+), 10738 deletions(-) BREAKING CHANGE: Agent file paths now use category structure. Update references from .opencode/agent/openagent.md to .opencode/agent/core/openagent.md. Eval framework maintains backward compatibility via path resolution. * refactor(evals): consolidate documentation and enhance test infrastructure - Remove temporary project tracking files (PHASE_5_COMPLETE, PROJECT_COMPLETE, etc.) - Consolidate evaluation framework docs into main README - Enhance test execution with improved logging and multi-prompt support - Move system-builder from core to meta category - Add comprehensive test suites for openagent with organized structure - Create evaluation test structure for all subagents - Clean up archived workflows and redundant documentation - Update registry to reflect new agent organization - Add shared test templates and golden test patterns * feat(ci): add manual trigger support for bot-created PRs in validate-registry workflow - Add pr_number input to workflow_dispatch for manually triggering validation - Fetch PR details dynamically when triggered manually - Support both automatic (PR event) and manual (workflow_dispatch) triggers - Enable validation of bot-created PRs like automated version bumps - Update branch detection and push logic to handle both trigger types - Add documentation explaining how to manually trigger for bot PRs * feat(evals): add explicit context file validation to test framework - Add expectedContextFiles field to test YAML schema for explicit context file specification - Enhance context-loading-evaluator to support both auto-detect and explicit validation modes - Update documentation with comprehensive guide and examples - Clean up archived legacy test structure (90+ old test files) - Add new example tests demonstrating explicit context validation - Backward compatible with existing tests * feat(evals): add multi-agent logging system and performance optimizations Implement comprehensive multi-agent logging and performance improvements for the eval framework. Task 01: Multi-Agent Logging System - Add complete hierarchical logging module (evals/framework/src/logging/) - SessionTracker: tracks parent-child delegation hierarchies - MultiAgentLogger: pretty-prints logs with visual indentation - Formatters: box characters and emoji formatting - 37 passing unit tests (session-tracker, logger, integration) - Integrate with SDK event stream handler - Hook into session.created, message.updated, message.part.updated events - Real-time child session detection via timestamp heuristics - Message deduplication for cleaner output - Enable in debug mode only (<1% performance overhead) - Add demo script and comprehensive documentation Task 02: Performance Optimizations - Reduce grace period from 5s to 2s (67% reduction, 10-20% faster tests) - Add PerformanceMetricsEvaluator for bottleneck identification - Collects tool latencies, inference time, idle time - Provides full performance visibility - Update smoke test to validate delegation (core feature) - Replace simple read test with multi-agent delegation test - Tests both parent and child agent functionality - Validates multi-agent logging system Results: - 37 unit tests passing - Smoke test passing (95/100 score) - Clean hierarchical logging output - 10-20% faster test execution - Full multi-agent visibility Files changed: 16 files, +2678 lines Time saved: ~6-8 days (completed in 6 hours vs 3-5 day estimate) * feat(evals): show child agent execution in non-debug mode - Enable MultiAgentLogger in both debug and non-debug modes - Add verbose parameter to control output level - Non-verbose mode shows concise child session lifecycle: - Child agent started message - Child agent completed with duration - Verbose mode (--debug) shows full delegation hierarchy - Update documentation to reflect new behavior This provides visibility into delegation without overwhelming output, giving confidence that child agents are actually running. * feat(evals): improve delegation testing and behavior validation Major improvements to eval framework: Test Suite: - Add 00-smoke-test.yaml (basic read operation) - Rename 01-smoke-test.yaml → 02-delegation-test.yaml (delegation test) - Add simple-responder test agent for delegation testing - Add debug test: simple-subagent-call.yaml Evaluators: - Add agent-model-evaluator.ts for agent/model validation - Enhance behavior-evaluator.ts with detailed task tool output - Improve delegation-evaluator.ts with better evidence tracking Schema & Execution: - Add expectedAgent and expectedModel to test schema - Improve test executor with better logging and error handling Documentation: - Update evals/README.md with new features and performance improvements - Remove outdated MULTI_AGENT_LOGGING_COMPLETE.md Registry: - Add simple-responder test agent to registry Fixes: - Fix dashboard.sh path resolution These changes provide better visibility into delegation, improved test coverage, and clearer validation of agent behavior. * chore: add GitHub Actions workflow and plugin documentation - Add .github/workflows/evals/run-evaluations.yml for automated eval testing - Add dev/ai-tools/opencode/plugins/Plugin-inspiration.md for plugin development reference |
||
|
darrenhinde
|
c6775bbc40 |
chore: consolidate development changes and refactor agent system
Major Changes: - Rename codebase-agent to opencoder for clarity - Add agent validator plugin with comprehensive testing - Refactor prompt engineering commands (enhancer → optimizer) - Update task-manager subagent with improved workflows - Enhance plugin system with validation and documentation Agent System: - Renamed: codebase-agent.md → opencoder.md - Updated: task-manager.md with better delegation patterns - Added: opencoder.md documentation Plugin System: - New: agent-validator.ts for prompt validation - New: VALIDATOR_GUIDE.md with usage instructions - New: Test suite (test-validator.sh, test-validation.sh) - Updated: notify.ts, telegram-notify.ts with improvements - Updated: package.json, bun.lock with new dependencies Commands: - Renamed: prompt-enchancer.md → prompt-engineering/prompt-enhancer.md - New: prompt-engineering/prompt-optimizer.md - Updated: build-context-system.md, validate-repo.md Documentation: - Updated: README.md with latest features - Updated: openagent.md documentation - New: opencoder.md agent documentation - Updated: agent-system-blueprint.md - Updated: system-builder docs (README, quick-start) - Updated: installation.md, collision-handling.md Registry: - Updated: registry.json with new components This consolidates all development work for the upcoming release. |
||
|
darrenhinde
|
ca323285b2 |
Rename to OpenAgents, remove model specs, and update documentation
Major Changes: - Rename repository from opencode-agents to OpenAgents - Remove all hardcoded model specifications (now model-agnostic) - Fix documentation: profile names, component counts, add Business profile - Add future plans note for other AI coding tools Repository Rename: - Update all URLs to github.com/darrenhinde/OpenAgents - Update git remote, README, install scripts, documentation - 15 files updated with new repository name Model-Agnostic Framework: - Remove model specifications from 6 agent files - Framework now works with any AI provider (Claude, GPT, Gemini, local models) - Users control model selection via OpenCode CLI settings Documentation Fixes: - Correct profile name: Core → Essential - Add missing Business profile documentation - Fix component counts: Developer (19), Advanced (32) - Document all 5 profiles with accurate descriptions Future Plans: - Add note about support for other AI tools (Cursor, Claude Code, etc.) - Clear focus on stabilizing OpenCode integration first This makes OpenAgents a truly universal, model-agnostic framework accessible to all users regardless of their AI provider preference. |
||
|
darrenhinde
|
5f2cec1404 |
✨ feat: introduce OpenAgent universal agent with session management
Major architectural refactoring: - Add OpenAgent as universal primary agent for questions, tasks, and workflows - Implement session-based context management with manifest tracking - Reorganize subagents into logical categories (code/core/utils) - Add dynamic context discovery and loading system - Introduce cleanup script for stale sessions - Update registry with new categorization (essential/standard/extended/specialized/meta) - Rename 'core' profile to 'essential' and add 'business' profile - Add comprehensive documentation with visual diagrams - Update installation script with new profiles Breaking changes: - Subagent paths changed from analysis/core to code/core/utils - Profile 'core' renamed to 'essential' (backward compatible) New features: - Session isolation for concurrent agent instances (.tmp/sessions/) - Automatic stale session cleanup (>24 hours) - Manifest-based context indexing and discovery - Business profile for content-focused workflows - OpenAgent with plan-approve-execute-validate-summarize workflow Components: 32 total (3 agents, 13 subagents, 8 commands, 2 tools, 2 plugins, 2 contexts, 2 config) Documentation: - docs/agents/openagent.md - Comprehensive guide with Mermaid diagrams - docs/guides/building-with-opencode.md - Power user formula - scripts/README.md - Script documentation - .opencode/context/core/context-management.md - Session management strategy |
Renamed from .opencode/agent/task-manager.md (Browse further)