OpenWriterControle/.github/workflows/validate-registry.yml
Darren Hinde 784ffadf92
chore: verify and stabilize main branch (#42)
* chore: update package-lock.json after npm install

* chore: remove unused ajv-cli, keep vitest at stable 1.6.1

- Removed ajv-cli (unused dev dependency, had high severity vulnerability)
- Kept vitest at 1.6.1 (stable, upgrading to v4 requires extensive refactoring)
- Reduced vulnerabilities from 6 to 4 (all moderate, dev-only)
- All remaining vulnerabilities are in esbuild/vite (dev server only, no production impact)
- Build and tests verified working

* feat(evals): add execution-balance validation to 21 OpenAgent tests and enhance agent system

Major improvements to evaluation framework and agent capabilities:

## Execution Balance Evaluator Integration
- Added execution-balance rule to 21 OpenAgent tests (2.7% → 31.5% coverage)
- Validates "read before execute" pattern across critical rules, workflows, and edge cases
- Tests now check: read operations before execution tools, healthy read/exec ratio (≥1.0)
- Coverage includes: approval-gate, context-loading, stop-on-failure, report-first tests

## Agent System Enhancements
- Enhanced OpenCoder agent with critical rules enforcement (approval gates, context loading)
- Added structured delegation rules and workflow phases
- Improved error handling with REPORT→PROPOSE→REQUEST→FIX pattern

## New Agent Creation System
- Added research-backed agent creation templates and guides
- Includes 8 test templates covering planning, context, incremental execution, tools
- Automated agent scaffolding with /create-agent command
- Complete documentation in .opencode/command/openagents/new-agents/

## OpenCoder Test Suite
- Added 8 comprehensive test cases for OpenCoder agent
- Tests cover: planning, context-loading, delegation, error-handling, implementation
- Includes DEBUG_GUIDE.md and QUICK_TEST_GUIDE.md for developers

## Documentation & Tooling
- Added DEVELOPMENT.md guide for contributors
- Enhanced CONTRIBUTING.md with quick reference and common commands
- Improved event-logger with DEBUG_VERBOSE mode for detailed output
- Added debug scripts: show-test-conversation.sh, run-test-verbose.sh
- Removed deprecated EXTERNAL_PR_GUIDE.md

## Version Bump
- Bumped version to 0.0.3 (patch)

Files changed: 60+ files (21 test updates, 16 new agent templates, 8 new tests, docs, tooling)

* feat(ci): add PR title validation for semantic versioning

Add automated PR title validation to ensure conventional commit format.
This enables proper automatic version bumping when PRs are merged.

Features:
- Validates PR titles against conventional commit patterns
- Shows expected version bump (major/minor/patch)
- Posts helpful comment with examples if validation fails
- Supports all conventional commit types (feat, fix, docs, etc.)
- Supports breaking changes (feat!, fix!)
- Supports pre-release tags ([alpha], [beta], [rc])

When PR titles follow the format, version bumping works correctly:
- feat: → minor bump (0.3.0 → 0.4.0)
- fix: → patch bump (0.3.0 → 0.3.1)
- feat!: → major bump (0.3.0 → 1.0.0)

* fix(ci): improve PR checks workflow with sequential execution

Reorganize PR checks to run in logical sequence and skip unnecessary checks:

Changes:
- Run PR title validation first (fast, always required)
- Detect changed files to determine which checks to run
- Only run build checks if evals/ files changed
- Add comprehensive summary job showing all check results
- Prevent cache errors when evals files not changed

Benefits:
- Faster PR checks (skip unnecessary builds)
- Clear sequential execution (title → changes → build)
- Better error messages and summaries
- Reduced CI/CD costs

* fix(ci): resolve PR #42 issues and refactor prompt architecture

## Fixed PR #42 Issues

1. **Prompt Validation** - Refactored to new architecture where agent files are canonical defaults
2. **Package-lock.json** - Fixed workflow to use root package-lock (npm workspaces)
3. **Git Merge Base** - Added fetch-depth: 0 to PR checks workflow

## New Prompt Architecture

**Before:**
- Agent files had to match .opencode/prompts/<agent>/default.md
- Caused validation failures when updating prompts
- Redundant duplication

**After:**
- Agent files (.opencode/agent/*.md) = Canonical defaults (source of truth)
- Prompt variants (.opencode/prompts/<agent>/<model>.md) = Model-specific optimizations
- No more default.md files needed

## Changes

### Workflows
- pr-checks.yml: Added fetch-depth: 0 for full git history
- validate-test-suites.yml: Fixed package-lock path for npm workspaces
- validate-registry.yml: Updated validation messaging

### Scripts
- validate-pr.sh: Validates prompt library structure (rejects default.md files)
- use-prompt.sh: Handles 'default' as agent file, variants as model-specific
- test-prompt.sh: Tests default variant correctly, saves results to prompts/results/

### Documentation
- .opencode/prompts/README.md: Updated architecture explanation
- docs/contributing/CONTRIBUTING.md: Updated prompt workflow
- scripts/development/demo.sh: Updated structure display

## Results

- 12 files changed, 231 insertions(+), 811 deletions(-)
- Net reduction: 580 lines (cleaner codebase)
- All validations passing
- Results directory structure preserved
- Backwards compatible with existing test results
2025-12-09 13:26:52 +00:00

259 lines
11 KiB
YAML
Raw Blame History

This file contains invisible Unicode characters

This file contains invisible Unicode characters that are indistinguishable to humans but may be processed differently by a computer. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

name: Validate Registry on PR
on:
pull_request:
branches:
- main
- dev
# Removed paths filter - this check is required by repository ruleset
# so it must run on ALL PRs to prevent blocking merges
workflow_dispatch:
inputs:
skip_validation:
description: 'Skip validation checks (maintainer override)'
required: false
type: boolean
default: false
permissions:
contents: write
pull-requests: write
jobs:
validate-and-update:
runs-on: ubuntu-latest
steps:
- name: Checkout PR branch
uses: actions/checkout@v4
with:
# For forks: fetch from fork repo, for internal: use head_ref
repository: ${{ github.event.pull_request.head.repo.full_name }}
ref: ${{ github.event.pull_request.head.ref }}
fetch-depth: 0
token: ${{ secrets.GITHUB_TOKEN }}
- name: Detect fork PR
id: fork_check
run: |
if [ "${{ github.event.pull_request.head.repo.full_name }}" != "${{ github.repository }}" ]; then
echo "is_fork=true" >> $GITHUB_OUTPUT
echo "🔀 Fork PR detected from: ${{ github.event.pull_request.head.repo.full_name }}"
else
echo "is_fork=false" >> $GITHUB_OUTPUT
echo "📝 Internal PR detected"
fi
- name: Install dependencies
run: |
sudo apt-get update
sudo apt-get install -y jq
- name: Make scripts executable
run: |
chmod +x scripts/registry/validate-registry.sh
chmod +x scripts/registry/auto-detect-components.sh
chmod +x scripts/registry/register-component.sh
chmod +x scripts/prompts/validate-pr.sh
- name: Auto-detect new components
id: auto_detect
run: |
echo "## 🔍 Auto-Detection Results" >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY
# Run auto-detect in dry-run mode first to see what would be added
if ./scripts/registry/auto-detect-components.sh --dry-run > /tmp/detect-output.txt 2>&1; then
cat /tmp/detect-output.txt >> $GITHUB_STEP_SUMMARY
# Check if new components were found
if grep -q "Found.*new component" /tmp/detect-output.txt; then
echo "new_components=true" >> $GITHUB_OUTPUT
echo "" >> $GITHUB_STEP_SUMMARY
echo "⚠️ New components detected - will auto-add to registry" >> $GITHUB_STEP_SUMMARY
else
echo "new_components=false" >> $GITHUB_OUTPUT
echo "✅ No new components found" >> $GITHUB_STEP_SUMMARY
fi
else
echo "new_components=false" >> $GITHUB_OUTPUT
echo "❌ Auto-detection failed" >> $GITHUB_STEP_SUMMARY
fi
- name: Add new components to registry
if: steps.auto_detect.outputs.new_components == 'true'
run: |
echo "## 📝 Adding New Components" >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY
./scripts/registry/auto-detect-components.sh --auto-add | tee -a $GITHUB_STEP_SUMMARY
- name: Validate prompt library structure
id: validate_prompts
run: |
echo "## 🔍 Prompt Library Validation" >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY
if ./scripts/prompts/validate-pr.sh > /tmp/prompt-validation.txt 2>&1; then
echo "prompt_validation=passed" >> $GITHUB_OUTPUT
echo "✅ Prompt library structure is valid!" >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY
echo "\`\`\`" >> $GITHUB_STEP_SUMMARY
cat /tmp/prompt-validation.txt >> $GITHUB_STEP_SUMMARY
echo "\`\`\`" >> $GITHUB_STEP_SUMMARY
else
echo "prompt_validation=failed" >> $GITHUB_OUTPUT
echo "❌ Prompt library validation failed!" >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY
echo "\`\`\`" >> $GITHUB_STEP_SUMMARY
cat /tmp/prompt-validation.txt >> $GITHUB_STEP_SUMMARY
echo "\`\`\`" >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY
echo "**Architecture:**" >> $GITHUB_STEP_SUMMARY
echo "- Agent files (.opencode/agent/*.md) = Canonical defaults" >> $GITHUB_STEP_SUMMARY
echo "- Prompt variants (.opencode/prompts/<agent>/<model>.md) = Model-specific" >> $GITHUB_STEP_SUMMARY
echo "- default.md files should NOT exist" >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY
echo "See [CONTRIBUTING.md](docs/contributing/CONTRIBUTING.md) for details" >> $GITHUB_STEP_SUMMARY
exit 1
fi
- name: Validate registry
id: validate
run: |
echo "## ✅ Registry Validation" >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY
if ./scripts/registry/validate-registry.sh -v > /tmp/validation-output.txt 2>&1; then
echo "validation=passed" >> $GITHUB_OUTPUT
echo "✅ All registry paths are valid!" >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY
echo "\`\`\`" >> $GITHUB_STEP_SUMMARY
tail -20 /tmp/validation-output.txt >> $GITHUB_STEP_SUMMARY
echo "\`\`\`" >> $GITHUB_STEP_SUMMARY
else
echo "validation=failed" >> $GITHUB_OUTPUT
echo "❌ Registry validation failed!" >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY
echo "\`\`\`" >> $GITHUB_STEP_SUMMARY
cat /tmp/validation-output.txt >> $GITHUB_STEP_SUMMARY
echo "\`\`\`" >> $GITHUB_STEP_SUMMARY
exit 1
fi
- name: Commit registry updates (Internal PRs only)
if: |
steps.fork_check.outputs.is_fork == 'false' &&
steps.auto_detect.outputs.new_components == 'true' &&
steps.validate_prompts.outputs.prompt_validation == 'passed' &&
steps.validate.outputs.validation == 'passed'
run: |
git config --local user.email "github-actions[bot]@users.noreply.github.com"
git config --local user.name "github-actions[bot]"
if ! git diff --quiet registry.json; then
git add registry.json
git commit -m "chore: auto-update registry with new components [skip ci]"
git push origin ${{ github.event.pull_request.head.ref }}
echo "## 🚀 Registry Updated" >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY
echo "Registry has been automatically updated with new components." >> $GITHUB_STEP_SUMMARY
echo "Changes have been pushed to this PR branch." >> $GITHUB_STEP_SUMMARY
else
echo "## ℹ️ No Changes to Commit" >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY
echo "Registry is already up to date." >> $GITHUB_STEP_SUMMARY
fi
- name: Fork PR notice
if: |
steps.fork_check.outputs.is_fork == 'true' &&
steps.auto_detect.outputs.new_components == 'true' &&
steps.validate_prompts.outputs.prompt_validation == 'passed' &&
steps.validate.outputs.validation == 'passed'
uses: actions/github-script@v7
with:
script: |
github.rest.issues.createComment({
issue_number: context.issue.number,
owner: context.repo.owner,
repo: context.repo.repo,
body: `## 📝 Registry Update Needed
Hi @${{ github.event.pull_request.user.login }}! 👋
New components were detected in your PR. Since this is a fork PR, I can't auto-commit the registry updates for security reasons.
**Please run these commands locally:**
\`\`\`bash
./scripts/registry/auto-detect-components.sh --auto-add
git add registry.json
git commit -m "chore: update registry"
git push
\`\`\`
Once you push the updated registry, the checks will pass! ✅`
});
- name: Fork PR summary
if: steps.fork_check.outputs.is_fork == 'true'
run: |
echo "## 🔀 Fork PR Detected" >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY
echo "This is an external contribution - thank you! 🎉" >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY
if [ "${{ steps.auto_detect.outputs.new_components }}" == "true" ]; then
echo "⚠️ **Action Required:** New components detected" >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY
echo "A comment has been posted with instructions to update the registry." >> $GITHUB_STEP_SUMMARY
else
echo "✅ No registry updates needed" >> $GITHUB_STEP_SUMMARY
fi
- name: Post validation summary
if: always()
run: |
echo "" >> $GITHUB_STEP_SUMMARY
echo "---" >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY
PROMPT_VALID="${{ steps.validate_prompts.outputs.prompt_validation }}"
REGISTRY_VALID="${{ steps.validate.outputs.validation }}"
if [ "$PROMPT_VALID" = "passed" ] && [ "$REGISTRY_VALID" = "passed" ]; then
echo "### ✅ All Validations Passed" >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY
echo "- ✅ Prompt library structure is valid" >> $GITHUB_STEP_SUMMARY
echo "- ✅ Registry paths are valid" >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY
echo "This PR is ready for review!" >> $GITHUB_STEP_SUMMARY
else
echo "### ❌ Validation Failed" >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY
if [ "$PROMPT_VALID" != "passed" ]; then
echo "- ❌ Prompt library validation failed" >> $GITHUB_STEP_SUMMARY
else
echo "- ✅ Prompt library validation passed" >> $GITHUB_STEP_SUMMARY
fi
if [ "$REGISTRY_VALID" != "passed" ]; then
echo "- ❌ Registry validation failed" >> $GITHUB_STEP_SUMMARY
else
echo "- ✅ Registry validation passed" >> $GITHUB_STEP_SUMMARY
fi
echo "" >> $GITHUB_STEP_SUMMARY
echo "Please fix the issues above before merging." >> $GITHUB_STEP_SUMMARY
fi
- name: Fail if validation failed
if: |
(steps.validate_prompts.outputs.prompt_validation == 'failed' || steps.validate.outputs.validation == 'failed') &&
github.event.inputs.skip_validation != 'true'
run: |
echo "❌ Validation failed - blocking PR merge"
echo "Maintainer can override by running workflow manually with 'skip_validation' enabled"
exit 1