OpenWriterControle/scripts
Darren Hinde 880396d37e
fix(registry): add missing agents to installation profiles (#64) (#66)
* Improve LLM integration tests: replace 'always pass' tests with meaningful validation

- Replace old llm-agent-behavior.test.ts (14 tests that always passed) with new llm-integration.test.ts (10 tests that can actually fail)
- Remove redundant tests already covered by unit tests (bash antipatterns, auto-fix detection)
- Add behavior-based validation using framework's built-in expectations (requiresApproval, mustUseDedicatedTools, requiresContext)
- Improve test resilience with graceful timeout handling for LLM unpredictability
- Add comprehensive validation report (LLM_INTEGRATION_VALIDATION.md)
- Remove outdated documentation (LLM_AGENT_TESTING.md, LLM_TEST_SUITE.md, PHASE_3_4_5_SUMMARY.md, COMPREHENSIVE_TEST_REPORT.md)

Test Results:
- All 10 LLM integration tests passing (42s execution time, 25% faster)
- Total test suite: 317 tests, 316 passing (99.7%)
- Tests proven to catch real issues (failed during development)
- Reviewer approved: Production ready (9.5/10 confidence)

What Changed:
- Tests can now actually fail (vs always passing)
- Use behavior expectations instead of forcing violations
- Focus on integration testing, not violation detection
- Honest about purpose and limitations

* fix(registry): add missing agents to installation profiles (#64)

- Add development agents (frontend-specialist, backend-specialist, devops-specialist, codebase-agent) to developer profile
- Add content agents (copywriter, technical-writer) and data-analyst to business profile
- Add all new agents to full and advanced profiles
- Add eval-runner and repo-manager to appropriate profiles
- Add context-retriever subagent to advanced profile

Create validation and documentation:
- Add profile coverage validation script (scripts/registry/validate-profile-coverage.sh)
- Add profile validation guide (.opencode/context/openagents-repo/guides/profile-validation.md)
- Add subagent invocation guide (.opencode/context/openagents-repo/guides/subagent-invocation.md)
- Document issue resolution (ISSUE_64_RESOLUTION.md)

Fixes #64 - Users installing with profiles now receive all agents added in v0.5.0
2025-12-29 23:46:12 +00:00
..
check-context-logs feat: add PR template and automated doc sync workflow (#40) 2025-12-08 22:27:01 +00:00
development refactor(evals): consolidate documentation and enhance test infrastructure (#56) 2025-12-18 09:52:11 +00:00
docs feat: add PR template and automated doc sync workflow (#40) 2025-12-08 22:27:01 +00:00
maintenance feat: add PR template and automated doc sync workflow (#40) 2025-12-08 22:27:01 +00:00
prompts refactor(evals): consolidate documentation and enhance test infrastructure (#56) 2025-12-18 09:52:11 +00:00
registry fix(registry): add missing agents to installation profiles (#64) (#66) 2025-12-29 23:46:12 +00:00
testing feat: add PR template and automated doc sync workflow (#40) 2025-12-08 22:27:01 +00:00
tests refactor: simplify agent execution philosophy and reorganize context structure 2025-11-19 13:13:06 +00:00
validation feat: add PR template and automated doc sync workflow (#40) 2025-12-08 22:27:01 +00:00
versioning feat: add PR template and automated doc sync workflow (#40) 2025-12-08 22:27:01 +00:00
README.md feat: add PR template and automated doc sync workflow (#40) 2025-12-08 22:27:01 +00:00

Scripts

This directory contains utility scripts for the OpenAgents system, organized by function.

Directory Structure

scripts/
├── registry/              # Component registry management
│   ├── auto-detect-components.sh
│   ├── register-component.sh
│   ├── validate-component.sh
│   └── validate-registry.sh
├── validation/            # Context and validation scripts
│   └── validate-context-refs.sh
├── versioning/            # Version management
│   └── bump-version.sh
├── maintenance/           # Cleanup and maintenance
│   ├── cleanup-stale-sessions.sh
│   └── uninstall.sh
├── development/           # Development utilities
│   ├── dashboard.sh
│   └── demo.sh
├── testing/               # Test runner
│   └── test.sh
├── check-context-logs/    # Context logging utilities
├── prompts/               # Prompt management
└── tests/                 # Installer test scripts

Available Scripts

Testing

  • testing/test.sh - Main test runner with multi-agent support
    • Run all tests: ./scripts/testing/test.sh openagent
    • Run core tests: ./scripts/testing/test.sh openagent --core (7 tests, ~5-8 min)
    • Run with specific model: ./scripts/testing/test.sh openagent opencode/grok-code-fast
    • Debug mode: ./scripts/testing/test.sh openagent --core --debug

See tests/ subdirectory for installer test scripts.

Registry Management

  • registry/auto-detect-components.sh - Auto-detect new components in .opencode/
  • registry/register-component.sh - Register a new component in the registry
  • registry/validate-component.sh - Validate component structure and metadata
  • registry/validate-registry.sh - Validate all registry paths

Validation

  • validation/validate-context-refs.sh - Validate context references in markdown files

Versioning

  • versioning/bump-version.sh - Bump version (alpha, beta, rc, patch, minor, major)

Maintenance

  • maintenance/cleanup-stale-sessions.sh - Remove stale agent sessions older than 24 hours
  • maintenance/uninstall.sh - Uninstall OpenCode agents

Development

  • development/demo.sh - Interactive demo of repository structure
  • development/dashboard.sh - Launch test results dashboard

Session Cleanup

Agent instances create temporary context files in .tmp/sessions/{session-id}/ for subagent delegation. These sessions are automatically cleaned up, but you can manually remove stale sessions:

# Clean up sessions older than 24 hours
./scripts/maintenance/cleanup-stale-sessions.sh

# Or manually delete all sessions
rm -rf .tmp/sessions/

Sessions are safe to delete anytime - they only contain temporary context files for agent coordination.

Usage Examples

Run Tests

# Run core test suite (fast, 7 tests, ~5-8 min)
./scripts/testing/test.sh openagent --core

# Run all tests for OpenAgent
./scripts/testing/test.sh openagent

# Run tests with specific model
./scripts/testing/test.sh openagent anthropic/claude-sonnet-4-5

# Run core tests with debug mode
./scripts/testing/test.sh openagent --core --debug

Registry Management

# Auto-detect new components
./scripts/registry/auto-detect-components.sh --dry-run

# Add new components to registry
./scripts/registry/auto-detect-components.sh --auto-add

# Validate registry
./scripts/registry/validate-registry.sh -v

# Register a specific component
./scripts/registry/register-component.sh path/to/component

# Validate a component
./scripts/registry/validate-component.sh path/to/component

Maintenance

# Clean stale sessions
./scripts/maintenance/cleanup-stale-sessions.sh

# Uninstall OpenCode agents
./scripts/maintenance/uninstall.sh

Development

# Run interactive demo
./scripts/development/demo.sh

# Launch test results dashboard
./scripts/development/dashboard.sh

Versioning

# Bump version
./scripts/versioning/bump-version.sh patch
./scripts/versioning/bump-version.sh minor
./scripts/versioning/bump-version.sh major
./scripts/versioning/bump-version.sh alpha

Core Test Suite

The core test suite is a subset of 7 carefully selected tests that provide ~85% coverage of critical OpenAgent functionality in just 5-8 minutes.

Why Use Core Tests?

  • ✅ Fast feedback - 5-8 minutes vs 40-80 minutes for full suite
  • ✅ Prompt iteration - Quick validation when updating agent prompts
  • ✅ Development - Fast validation during development cycles
  • ✅ Pre-commit - Quick checks before committing changes

What's Covered?

  1. Approval Gate - Critical safety rule
  2. Context Loading (Simple) - Most common use case
  3. Context Loading (Multi-Turn) - Complex scenarios
  4. Stop on Failure - Error handling
  5. Simple Task - No unnecessary delegation
  6. Subagent Delegation - Proper delegation when needed
  7. Tool Usage - Best practices

When to Use Full Suite?

Use the full test suite (71 tests) for:

  • 🔬 Release validation
  • 🔬 Comprehensive testing
  • 🔬 Edge case coverage
  • 🔬 Regression testing

See evals/agents/openagent/CORE_TESTS.md for detailed documentation.