* feat(agents): implement category-based agent organization system - Organize agents into domain categories (core, development, content, data, product, learning) - Move 10 agents to category subdirectories with proper git rename tracking - Update 13 subagents with category and type metadata in frontmatter - Add category metadata files (0-category.json) documenting common patterns - Implement local registry fallback in install script for offline development - Add comprehensive validation suite with 15 automated tests (100% pass rate) - Enhance registry validation with duplicate ID and consistency checks - Update eval framework with intelligent path resolution (backward compatible) - Archive legacy eval structure to _archive/ for reference - Update all documentation to reflect category-based structure - Bump version to 0.5.0 with accurate CHANGELOG Technical Details: - 23 agents organized (10 category agents, 13 subagents) - 6 category directories created - Path resolution supports both agent IDs and category paths - Registry schema updated to v2.0.0 - 159 files changed, 872 insertions(+), 10738 deletions(-) BREAKING CHANGE: Agent file paths now use category structure. Update references from .opencode/agent/openagent.md to .opencode/agent/core/openagent.md. Eval framework maintains backward compatibility via path resolution. * refactor(evals): consolidate documentation and enhance test infrastructure - Remove temporary project tracking files (PHASE_5_COMPLETE, PROJECT_COMPLETE, etc.) - Consolidate evaluation framework docs into main README - Enhance test execution with improved logging and multi-prompt support - Move system-builder from core to meta category - Add comprehensive test suites for openagent with organized structure - Create evaluation test structure for all subagents - Clean up archived workflows and redundant documentation - Update registry to reflect new agent organization - Add shared test templates and golden test patterns * feat(ci): add manual trigger support for bot-created PRs in validate-registry workflow - Add pr_number input to workflow_dispatch for manually triggering validation - Fetch PR details dynamically when triggered manually - Support both automatic (PR event) and manual (workflow_dispatch) triggers - Enable validation of bot-created PRs like automated version bumps - Update branch detection and push logic to handle both trigger types - Add documentation explaining how to manually trigger for bot PRs * feat(evals): add explicit context file validation to test framework - Add expectedContextFiles field to test YAML schema for explicit context file specification - Enhance context-loading-evaluator to support both auto-detect and explicit validation modes - Update documentation with comprehensive guide and examples - Clean up archived legacy test structure (90+ old test files) - Add new example tests demonstrating explicit context validation - Backward compatible with existing tests * feat(evals): add multi-agent logging system and performance optimizations Implement comprehensive multi-agent logging and performance improvements for the eval framework. Task 01: Multi-Agent Logging System - Add complete hierarchical logging module (evals/framework/src/logging/) - SessionTracker: tracks parent-child delegation hierarchies - MultiAgentLogger: pretty-prints logs with visual indentation - Formatters: box characters and emoji formatting - 37 passing unit tests (session-tracker, logger, integration) - Integrate with SDK event stream handler - Hook into session.created, message.updated, message.part.updated events - Real-time child session detection via timestamp heuristics - Message deduplication for cleaner output - Enable in debug mode only (<1% performance overhead) - Add demo script and comprehensive documentation Task 02: Performance Optimizations - Reduce grace period from 5s to 2s (67% reduction, 10-20% faster tests) - Add PerformanceMetricsEvaluator for bottleneck identification - Collects tool latencies, inference time, idle time - Provides full performance visibility - Update smoke test to validate delegation (core feature) - Replace simple read test with multi-agent delegation test - Tests both parent and child agent functionality - Validates multi-agent logging system Results: - 37 unit tests passing - Smoke test passing (95/100 score) - Clean hierarchical logging output - 10-20% faster test execution - Full multi-agent visibility Files changed: 16 files, +2678 lines Time saved: ~6-8 days (completed in 6 hours vs 3-5 day estimate) * feat(evals): show child agent execution in non-debug mode - Enable MultiAgentLogger in both debug and non-debug modes - Add verbose parameter to control output level - Non-verbose mode shows concise child session lifecycle: - Child agent started message - Child agent completed with duration - Verbose mode (--debug) shows full delegation hierarchy - Update documentation to reflect new behavior This provides visibility into delegation without overwhelming output, giving confidence that child agents are actually running. * feat(evals): improve delegation testing and behavior validation Major improvements to eval framework: Test Suite: - Add 00-smoke-test.yaml (basic read operation) - Rename 01-smoke-test.yaml → 02-delegation-test.yaml (delegation test) - Add simple-responder test agent for delegation testing - Add debug test: simple-subagent-call.yaml Evaluators: - Add agent-model-evaluator.ts for agent/model validation - Enhance behavior-evaluator.ts with detailed task tool output - Improve delegation-evaluator.ts with better evidence tracking Schema & Execution: - Add expectedAgent and expectedModel to test schema - Improve test executor with better logging and error handling Documentation: - Update evals/README.md with new features and performance improvements - Remove outdated MULTI_AGENT_LOGGING_COMPLETE.md Registry: - Add simple-responder test agent to registry Fixes: - Fix dashboard.sh path resolution These changes provide better visibility into delegation, improved test coverage, and clearer validation of agent behavior. * chore: add GitHub Actions workflow and plugin documentation - Add .github/workflows/evals/run-evaluations.yml for automated eval testing - Add dev/ai-tools/opencode/plugins/Plugin-inspiration.md for plugin development reference
417 lines
11 KiB
Bash
Executable file
417 lines
11 KiB
Bash
Executable file
#!/bin/bash
|
||
# OpenCode Repository Demo
|
||
# Shows repository structure and prompt library system
|
||
#
|
||
# Usage:
|
||
# ./scripts/development/demo.sh # Interactive mode
|
||
# ./scripts/development/demo.sh --quick # Quick tour (non-interactive)
|
||
# ./scripts/development/demo.sh --full # Full demo (non-interactive)
|
||
|
||
set -e
|
||
|
||
# Colors
|
||
RED='\033[0;31m'
|
||
GREEN='\033[0;32m'
|
||
YELLOW='\033[1;33m'
|
||
BLUE='\033[0;34m'
|
||
PURPLE='\033[0;35m'
|
||
CYAN='\033[0;36m'
|
||
NC='\033[0m' # No Color
|
||
|
||
# Helper functions
|
||
print_header() {
|
||
echo ""
|
||
echo -e "${CYAN}━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━${NC}"
|
||
echo -e "${CYAN} $1${NC}"
|
||
echo -e "${CYAN}━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━${NC}"
|
||
echo ""
|
||
}
|
||
|
||
print_section() {
|
||
echo ""
|
||
echo -e "${BLUE}▶ $1${NC}"
|
||
echo ""
|
||
}
|
||
|
||
print_success() {
|
||
echo -e "${GREEN}✓${NC} $1"
|
||
}
|
||
|
||
print_info() {
|
||
echo -e "${YELLOW}ℹ${NC} $1"
|
||
}
|
||
|
||
pause() {
|
||
if [ "$NON_INTERACTIVE" != "true" ]; then
|
||
echo ""
|
||
read -p "Press Enter to continue..."
|
||
else
|
||
echo ""
|
||
fi
|
||
}
|
||
|
||
# Main demo functions
|
||
show_welcome() {
|
||
clear
|
||
print_header "Welcome to OpenCode Agents"
|
||
|
||
cat << EOF
|
||
OpenCode is a context-aware agent system for AI-powered development.
|
||
|
||
This demo will show you:
|
||
1. Repository structure
|
||
2. Agent system architecture
|
||
3. Prompt library system
|
||
4. Testing framework
|
||
5. How to contribute
|
||
|
||
Choose a mode:
|
||
[1] Quick Tour (2-3 minutes)
|
||
[2] Full Demo (10-15 minutes)
|
||
[3] Interactive Explorer
|
||
[q] Quit
|
||
|
||
EOF
|
||
|
||
read -p "Your choice: " choice
|
||
echo "$choice"
|
||
}
|
||
|
||
show_repo_structure() {
|
||
print_header "Repository Structure"
|
||
|
||
print_section "Main Directories"
|
||
|
||
cat << EOF
|
||
${GREEN}.opencode/${NC}
|
||
├── agent/ ${YELLOW}# Agents (category-based)${NC}
|
||
│ ├── core/
|
||
│ │ ├── openagent.md ${YELLOW}# Universal orchestrator${NC}
|
||
│ │ └── opencoder.md ${YELLOW}# Development specialist${NC}
|
||
│ ├── development/ ${YELLOW}# Development specialists${NC}
|
||
│ └── subagents/ ${YELLOW}# Specialized subagents${NC}
|
||
├── prompts/ ${YELLOW}# Prompt library (variants)${NC}
|
||
├── command/ ${YELLOW}# Slash commands${NC}
|
||
├── tool/ ${YELLOW}# Utility tools${NC}
|
||
└── context/ ${YELLOW}# Context files${NC}
|
||
|
||
${GREEN}evals/${NC}
|
||
├── agents/ ${YELLOW}# Agent test suites${NC}
|
||
├── framework/ ${YELLOW}# Testing framework${NC}
|
||
└── results/ ${YELLOW}# Test results${NC}
|
||
|
||
${GREEN}scripts/${NC}
|
||
├── prompts/ ${YELLOW}# Prompt management${NC}
|
||
└── tests/ ${YELLOW}# Test utilities${NC}
|
||
|
||
${GREEN}docs/${NC}
|
||
├── agents/ ${YELLOW}# Agent documentation${NC}
|
||
├── contributing/ ${YELLOW}# Contribution guides${NC}
|
||
└── guides/ ${YELLOW}# User guides${NC}
|
||
EOF
|
||
|
||
pause
|
||
}
|
||
|
||
show_agent_system() {
|
||
print_header "Agent System"
|
||
|
||
print_section "Main Agents"
|
||
|
||
if [ -f ".opencode/agent/core/openagent.md" ]; then
|
||
print_success "core/openagent.md - Universal orchestrator agent"
|
||
echo " Handles planning, delegation, and workflow management"
|
||
fi
|
||
|
||
if [ -f ".opencode/agent/core/opencoder.md" ]; then
|
||
print_success "core/opencoder.md - Development specialist"
|
||
echo " Focused on writing clean, maintainable code"
|
||
fi
|
||
|
||
print_section "Subagents"
|
||
|
||
if [ -d ".opencode/agent/subagents" ]; then
|
||
subagent_count=$(find .opencode/agent/subagents -name "*.md" -type f | wc -l | tr -d ' ')
|
||
echo "Found $subagent_count specialized subagents:"
|
||
echo ""
|
||
|
||
# Show subagent categories
|
||
for category in .opencode/agent/subagents/*/; do
|
||
if [ -d "$category" ]; then
|
||
category_name=$(basename "$category")
|
||
agent_count=$(find "$category" -name "*.md" -type f | wc -l | tr -d ' ')
|
||
echo -e "${GREEN}$category_name/${NC} ($agent_count agents)"
|
||
fi
|
||
done
|
||
fi
|
||
|
||
pause
|
||
}
|
||
|
||
show_prompt_library() {
|
||
print_header "Prompt Library System"
|
||
|
||
print_section "How It Works"
|
||
|
||
cat << EOF
|
||
The prompt library allows different AI models to use optimized prompts:
|
||
|
||
${GREEN}.opencode/agent/core/${NC}
|
||
└── openagent.md ${YELLOW}# Canonical default (Claude-optimized)${NC}
|
||
|
||
${GREEN}.opencode/prompts/core/openagent/${NC}
|
||
├── gpt.md ${YELLOW}# GPT-4 optimized${NC}
|
||
├── gemini.md ${YELLOW}# Gemini optimized${NC}
|
||
├── grok.md ${YELLOW}# Grok optimized${NC}
|
||
├── llama.md ${YELLOW}# Llama/OSS optimized${NC}
|
||
├── TEMPLATE.md ${YELLOW}# Template for new variants${NC}
|
||
└── results/ ${YELLOW}# Test results (all variants)${NC}
|
||
|
||
${BLUE}Key Principles:${NC}
|
||
• Agent files are canonical defaults (source of truth)
|
||
• Variants are model-specific optimizations
|
||
• All variants tested and documented
|
||
• Users can choose the best prompt for their model
|
||
• Contributors can add optimized variants
|
||
|
||
EOF
|
||
|
||
pause
|
||
|
||
print_section "Available Variants"
|
||
|
||
if [ -d ".opencode/prompts" ]; then
|
||
for agent_dir in .opencode/prompts/*/; do
|
||
if [ -d "$agent_dir" ]; then
|
||
agent=$(basename "$agent_dir")
|
||
echo -e "${GREEN}$agent:${NC}"
|
||
|
||
variant_count=0
|
||
for variant in "$agent_dir"*.md; do
|
||
if [ -f "$variant" ]; then
|
||
variant_name=$(basename "$variant" .md)
|
||
if [ "$variant_name" != "README" ] && [ "$variant_name" != "TEMPLATE" ]; then
|
||
variant_count=$((variant_count + 1))
|
||
# Check if results exist
|
||
results_file="$agent_dir/results/$variant_name-results.json"
|
||
if [ -f "$results_file" ]; then
|
||
passed=$(jq -r '.passed // 0' "$results_file" 2>/dev/null || echo "?")
|
||
total=$(jq -r '.total // 0' "$results_file" 2>/dev/null || echo "?")
|
||
echo " ✓ $variant_name ($passed/$total tests passing)"
|
||
else
|
||
echo " • $variant_name (not tested)"
|
||
fi
|
||
fi
|
||
fi
|
||
done
|
||
|
||
if [ $variant_count -eq 0 ]; then
|
||
echo " ${YELLOW}No variants yet - coming soon!${NC}"
|
||
fi
|
||
echo ""
|
||
fi
|
||
done
|
||
else
|
||
echo "${YELLOW}Prompt library not yet set up - coming soon!${NC}"
|
||
fi
|
||
|
||
pause
|
||
}
|
||
|
||
show_testing_framework() {
|
||
print_header "Testing Framework"
|
||
|
||
print_section "Test Structure"
|
||
|
||
cat << EOF
|
||
${GREEN}evals/agents/openagent/tests/${NC}
|
||
├── 01-critical-rules/ ${YELLOW}# Core behavior tests${NC}
|
||
├── 02-workflow-stages/ ${YELLOW}# Workflow validation${NC}
|
||
├── 03-delegation/ ${YELLOW}# Delegation logic${NC}
|
||
└── ...
|
||
|
||
Each test validates specific agent behaviors.
|
||
|
||
EOF
|
||
|
||
if [ -d "evals/agents/openagent/tests" ]; then
|
||
test_count=$(find evals/agents/openagent/tests -name "*.json" -type f | wc -l | tr -d ' ')
|
||
echo "Total tests: $test_count"
|
||
echo ""
|
||
fi
|
||
|
||
print_section "Running Tests"
|
||
|
||
cat << EOF
|
||
${BLUE}Test a prompt variant:${NC}
|
||
./scripts/prompts/test-prompt.sh core/openagent sonnet-4
|
||
|
||
${BLUE}Validate PR:${NC}
|
||
./scripts/prompts/validate-pr.sh
|
||
|
||
${BLUE}Use a variant:${NC}
|
||
./scripts/prompts/use-prompt.sh core/openagent sonnet-4
|
||
|
||
EOF
|
||
|
||
pause
|
||
}
|
||
|
||
show_contributing() {
|
||
print_header "Contributing"
|
||
|
||
print_section "How to Contribute"
|
||
|
||
cat << EOF
|
||
${BLUE}1. Create a prompt variant:${NC}
|
||
cp .opencode/prompts/core/openagent/TEMPLATE.md .opencode/prompts/core/openagent/my-variant.md
|
||
|
||
${BLUE}2. Edit your variant:${NC}
|
||
# Optimize for your target model
|
||
vim .opencode/prompts/core/openagent/my-variant.md
|
||
|
||
${BLUE}3. Test it:${NC}
|
||
./scripts/prompts/test-prompt.sh core/openagent my-variant
|
||
|
||
${BLUE}4. Document results:${NC}
|
||
# Update .opencode/prompts/core/openagent/README.md with test results
|
||
|
||
${BLUE}5. Submit PR:${NC}
|
||
# Include your variant and results
|
||
# Do NOT change the default prompt
|
||
|
||
${YELLOW}Important:${NC} All PRs must use default prompts (CI enforces this)
|
||
|
||
EOF
|
||
|
||
print_section "Other Ways to Contribute"
|
||
|
||
cat << EOF
|
||
• Add new agents or subagents
|
||
• Improve documentation
|
||
• Add test cases
|
||
• Fix bugs
|
||
• Enhance the testing framework
|
||
|
||
See ${BLUE}docs/contributing/CONTRIBUTING.md${NC} for details.
|
||
|
||
EOF
|
||
|
||
pause
|
||
}
|
||
|
||
interactive_mode() {
|
||
while true; do
|
||
clear
|
||
print_header "Interactive Explorer"
|
||
|
||
cat << EOF
|
||
Choose a section to explore:
|
||
[1] Repository Structure
|
||
[2] Agent System
|
||
[3] Prompt Library
|
||
[4] Testing Framework
|
||
[5] Contributing Guide
|
||
[b] Back to main menu
|
||
[q] Quit
|
||
|
||
EOF
|
||
|
||
read -p "Your choice: " choice
|
||
|
||
case $choice in
|
||
1) show_repo_structure ;;
|
||
2) show_agent_system ;;
|
||
3) show_prompt_library ;;
|
||
4) show_testing_framework ;;
|
||
5) show_contributing ;;
|
||
b) return ;;
|
||
q) exit 0 ;;
|
||
*) echo "Invalid choice" ;;
|
||
esac
|
||
done
|
||
}
|
||
|
||
quick_tour() {
|
||
show_repo_structure
|
||
show_agent_system
|
||
show_prompt_library
|
||
|
||
print_header "Quick Tour Complete!"
|
||
|
||
cat << EOF
|
||
${GREEN}Next Steps:${NC}
|
||
• Read the docs: ${BLUE}docs/README.md${NC}
|
||
• Try the agents: ${BLUE}.opencode/agent/${NC}
|
||
• Run tests: ${BLUE}./scripts/prompts/test-prompt.sh${NC}
|
||
• Contribute: ${BLUE}docs/contributing/CONTRIBUTING.md${NC}
|
||
|
||
${YELLOW}For more details, run:${NC}
|
||
./scripts/development/demo.sh
|
||
|
||
EOF
|
||
}
|
||
|
||
full_demo() {
|
||
show_repo_structure
|
||
show_agent_system
|
||
show_prompt_library
|
||
show_testing_framework
|
||
show_contributing
|
||
|
||
print_header "Demo Complete!"
|
||
|
||
cat << EOF
|
||
${GREEN}You've seen:${NC}
|
||
✓ Repository structure
|
||
✓ Agent system architecture
|
||
✓ Prompt library system
|
||
✓ Testing framework
|
||
✓ How to contribute
|
||
|
||
${YELLOW}Ready to get started?${NC}
|
||
• Read: ${BLUE}docs/getting-started/installation.md${NC}
|
||
• Explore: ${BLUE}.opencode/prompts/${NC}
|
||
• Test: ${BLUE}./scripts/prompts/test-prompt.sh${NC}
|
||
|
||
EOF
|
||
}
|
||
|
||
# Main
|
||
main() {
|
||
# Check for command line arguments
|
||
if [ "$1" = "--quick" ]; then
|
||
NON_INTERACTIVE=true
|
||
quick_tour
|
||
exit 0
|
||
elif [ "$1" = "--full" ]; then
|
||
NON_INTERACTIVE=true
|
||
full_demo
|
||
exit 0
|
||
elif [ "$1" = "--help" ] || [ "$1" = "-h" ]; then
|
||
cat << EOF
|
||
OpenCode Repository Demo
|
||
|
||
Usage:
|
||
./scripts/development/demo.sh Interactive mode with menu
|
||
./scripts/development/demo.sh --quick Quick tour (non-interactive)
|
||
./scripts/development/demo.sh --full Full demo (non-interactive)
|
||
./scripts/development/demo.sh --help Show this help
|
||
|
||
EOF
|
||
exit 0
|
||
fi
|
||
|
||
# Interactive mode
|
||
choice=$(show_welcome)
|
||
|
||
case $choice in
|
||
1) quick_tour ;;
|
||
2) full_demo ;;
|
||
3) interactive_mode ;;
|
||
q) exit 0 ;;
|
||
*) echo "Invalid choice"; exit 1 ;;
|
||
esac
|
||
}
|
||
|
||
main "$@"
|