784ffadf92
* chore: update package-lock.json after npm install * chore: remove unused ajv-cli, keep vitest at stable 1.6.1 - Removed ajv-cli (unused dev dependency, had high severity vulnerability) - Kept vitest at 1.6.1 (stable, upgrading to v4 requires extensive refactoring) - Reduced vulnerabilities from 6 to 4 (all moderate, dev-only) - All remaining vulnerabilities are in esbuild/vite (dev server only, no production impact) - Build and tests verified working * feat(evals): add execution-balance validation to 21 OpenAgent tests and enhance agent system Major improvements to evaluation framework and agent capabilities: ## Execution Balance Evaluator Integration - Added execution-balance rule to 21 OpenAgent tests (2.7% → 31.5% coverage) - Validates "read before execute" pattern across critical rules, workflows, and edge cases - Tests now check: read operations before execution tools, healthy read/exec ratio (≥1.0) - Coverage includes: approval-gate, context-loading, stop-on-failure, report-first tests ## Agent System Enhancements - Enhanced OpenCoder agent with critical rules enforcement (approval gates, context loading) - Added structured delegation rules and workflow phases - Improved error handling with REPORT→PROPOSE→REQUEST→FIX pattern ## New Agent Creation System - Added research-backed agent creation templates and guides - Includes 8 test templates covering planning, context, incremental execution, tools - Automated agent scaffolding with /create-agent command - Complete documentation in .opencode/command/openagents/new-agents/ ## OpenCoder Test Suite - Added 8 comprehensive test cases for OpenCoder agent - Tests cover: planning, context-loading, delegation, error-handling, implementation - Includes DEBUG_GUIDE.md and QUICK_TEST_GUIDE.md for developers ## Documentation & Tooling - Added DEVELOPMENT.md guide for contributors - Enhanced CONTRIBUTING.md with quick reference and common commands - Improved event-logger with DEBUG_VERBOSE mode for detailed output - Added debug scripts: show-test-conversation.sh, run-test-verbose.sh - Removed deprecated EXTERNAL_PR_GUIDE.md ## Version Bump - Bumped version to 0.0.3 (patch) Files changed: 60+ files (21 test updates, 16 new agent templates, 8 new tests, docs, tooling) * feat(ci): add PR title validation for semantic versioning Add automated PR title validation to ensure conventional commit format. This enables proper automatic version bumping when PRs are merged. Features: - Validates PR titles against conventional commit patterns - Shows expected version bump (major/minor/patch) - Posts helpful comment with examples if validation fails - Supports all conventional commit types (feat, fix, docs, etc.) - Supports breaking changes (feat!, fix!) - Supports pre-release tags ([alpha], [beta], [rc]) When PR titles follow the format, version bumping works correctly: - feat: → minor bump (0.3.0 → 0.4.0) - fix: → patch bump (0.3.0 → 0.3.1) - feat!: → major bump (0.3.0 → 1.0.0) * fix(ci): improve PR checks workflow with sequential execution Reorganize PR checks to run in logical sequence and skip unnecessary checks: Changes: - Run PR title validation first (fast, always required) - Detect changed files to determine which checks to run - Only run build checks if evals/ files changed - Add comprehensive summary job showing all check results - Prevent cache errors when evals files not changed Benefits: - Faster PR checks (skip unnecessary builds) - Clear sequential execution (title → changes → build) - Better error messages and summaries - Reduced CI/CD costs * fix(ci): resolve PR #42 issues and refactor prompt architecture ## Fixed PR #42 Issues 1. **Prompt Validation** - Refactored to new architecture where agent files are canonical defaults 2. **Package-lock.json** - Fixed workflow to use root package-lock (npm workspaces) 3. **Git Merge Base** - Added fetch-depth: 0 to PR checks workflow ## New Prompt Architecture **Before:** - Agent files had to match .opencode/prompts/<agent>/default.md - Caused validation failures when updating prompts - Redundant duplication **After:** - Agent files (.opencode/agent/*.md) = Canonical defaults (source of truth) - Prompt variants (.opencode/prompts/<agent>/<model>.md) = Model-specific optimizations - No more default.md files needed ## Changes ### Workflows - pr-checks.yml: Added fetch-depth: 0 for full git history - validate-test-suites.yml: Fixed package-lock path for npm workspaces - validate-registry.yml: Updated validation messaging ### Scripts - validate-pr.sh: Validates prompt library structure (rejects default.md files) - use-prompt.sh: Handles 'default' as agent file, variants as model-specific - test-prompt.sh: Tests default variant correctly, saves results to prompts/results/ ### Documentation - .opencode/prompts/README.md: Updated architecture explanation - docs/contributing/CONTRIBUTING.md: Updated prompt workflow - scripts/development/demo.sh: Updated structure display ## Results - 12 files changed, 231 insertions(+), 811 deletions(-) - Net reduction: 580 lines (cleaner codebase) - All validations passing - Results directory structure preserved - Backwards compatible with existing test results
66 lines
4.1 KiB
JSON
66 lines
4.1 KiB
JSON
{
|
|
"name": "opencode-agents",
|
|
"version": "0.3.0",
|
|
"description": "OpenCode agent evaluation framework and test suites",
|
|
"private": true,
|
|
"workspaces": [
|
|
"evals/framework"
|
|
],
|
|
"scripts": {
|
|
"test": "npm run test:all",
|
|
"test:all": "cd evals/framework && npm run eval:sdk",
|
|
"test:core": "npm run test:openagent:core",
|
|
"test:openagent": "cd evals/framework && npm run eval:sdk -- --agent=openagent",
|
|
"test:openagent:core": "cd evals/framework && npm run eval:sdk:core -- --agent=openagent",
|
|
"test:opencoder": "cd evals/framework && npm run eval:sdk -- --agent=opencoder",
|
|
"test:openagent:grok": "npm run test:openagent -- --model=opencode/grok-code-fast",
|
|
"test:openagent:claude": "npm run test:openagent -- --model=anthropic/claude-3-5-sonnet-20241022",
|
|
"test:openagent:gpt4": "npm run test:openagent -- --model=openai/gpt-4-turbo",
|
|
"test:opencoder:grok": "npm run test:opencoder -- --model=opencode/grok-code-fast",
|
|
"test:opencoder:claude": "npm run test:opencoder -- --model=anthropic/claude-3-5-sonnet-20241022",
|
|
"test:opencoder:gpt4": "npm run test:opencoder -- --model=openai/gpt-4-turbo",
|
|
"test:all:grok": "npm run test:all -- --model=opencode/grok-code-fast",
|
|
"test:all:claude": "npm run test:all -- --model=anthropic/claude-3-5-sonnet-20241022",
|
|
"test:all:gpt4": "npm run test:all -- --model=openai/gpt-4-turbo",
|
|
"test:pattern": "cd evals/framework && npm run eval:sdk -- --pattern",
|
|
"test:debug": "cd evals/framework && npm run eval:sdk -- --debug",
|
|
"test:openagent:developer": "npm run test:openagent -- --pattern='developer/*.yaml'",
|
|
"test:openagent:context": "npm run test:openagent -- --pattern='context-loading/*.yaml'",
|
|
"test:openagent:business": "npm run test:openagent -- --pattern='business/*.yaml'",
|
|
"test:opencoder:developer": "npm run test:opencoder -- --pattern='developer/*.yaml'",
|
|
"test:opencoder:bash": "npm run test:opencoder -- --pattern='developer/bash-*.yaml'",
|
|
"test:ci": "npm run test:ci:openagent && npm run test:ci:opencoder",
|
|
"test:ci:openagent": "npm run test:openagent -- --pattern='smoke-test.yaml' --no-evaluators",
|
|
"test:ci:opencoder": "npm run test:opencoder -- --pattern='developer/simple-bash-test.yaml' --no-evaluators",
|
|
"dashboard": "cd evals/results && ./serve.sh",
|
|
"dashboard:open": "npm run dashboard && open http://localhost:8000",
|
|
"results:openagent": "echo 'OpenAgent results:' && ls -lh evals/results/history/*openagent*.json 2>/dev/null | tail -5 || echo 'No results yet'",
|
|
"results:opencoder": "echo 'OpenCoder results:' && ls -lh evals/results/history/*opencoder*.json 2>/dev/null | tail -5 || echo 'No results yet'",
|
|
"results:latest": "cat evals/results/latest.json 2>/dev/null | jq '.agent, .passed, .failed' || echo 'No results yet'",
|
|
"version": "node -p \"require('./package.json').version\"",
|
|
"version:bump": "./scripts/versioning/bump-version.sh",
|
|
"version:bump:patch": "npm version patch --no-git-tag-version && node -p \"require('./package.json').version\" > VERSION",
|
|
"version:bump:minor": "npm version minor --no-git-tag-version && node -p \"require('./package.json').version\" > VERSION",
|
|
"version:bump:major": "npm version major --no-git-tag-version && node -p \"require('./package.json').version\" > VERSION",
|
|
"version:bump:alpha": "npm version prerelease --preid=alpha --no-git-tag-version && node -p \"require('./package.json').version\" > VERSION",
|
|
"version:bump:beta": "npm version prerelease --preid=beta --no-git-tag-version && node -p \"require('./package.json').version\" > VERSION",
|
|
"version:bump:rc": "npm version prerelease --preid=rc --no-git-tag-version && node -p \"require('./package.json').version\" > VERSION",
|
|
"dev:setup": "cd evals/framework && npm install",
|
|
"dev:build": "cd evals/framework && npm run build",
|
|
"dev:test": "cd evals/framework && npm test",
|
|
"dev:clean": "cd evals/framework && rm -rf dist node_modules && npm install"
|
|
},
|
|
"keywords": [
|
|
"opencode",
|
|
"agents",
|
|
"evaluation",
|
|
"testing",
|
|
"ai"
|
|
],
|
|
"author": "Darren Hinde",
|
|
"license": "MIT",
|
|
"repository": {
|
|
"type": "git",
|
|
"url": "https://github.com/darrenhinde/OpenAgents.git"
|
|
}
|
|
}
|