OpenWriterControle/scripts/prompts/validate-pr.sh
Darren Hinde 784ffadf92
chore: verify and stabilize main branch (#42)
* chore: update package-lock.json after npm install

* chore: remove unused ajv-cli, keep vitest at stable 1.6.1

- Removed ajv-cli (unused dev dependency, had high severity vulnerability)
- Kept vitest at 1.6.1 (stable, upgrading to v4 requires extensive refactoring)
- Reduced vulnerabilities from 6 to 4 (all moderate, dev-only)
- All remaining vulnerabilities are in esbuild/vite (dev server only, no production impact)
- Build and tests verified working

* feat(evals): add execution-balance validation to 21 OpenAgent tests and enhance agent system

Major improvements to evaluation framework and agent capabilities:

## Execution Balance Evaluator Integration
- Added execution-balance rule to 21 OpenAgent tests (2.7% → 31.5% coverage)
- Validates "read before execute" pattern across critical rules, workflows, and edge cases
- Tests now check: read operations before execution tools, healthy read/exec ratio (≥1.0)
- Coverage includes: approval-gate, context-loading, stop-on-failure, report-first tests

## Agent System Enhancements
- Enhanced OpenCoder agent with critical rules enforcement (approval gates, context loading)
- Added structured delegation rules and workflow phases
- Improved error handling with REPORT→PROPOSE→REQUEST→FIX pattern

## New Agent Creation System
- Added research-backed agent creation templates and guides
- Includes 8 test templates covering planning, context, incremental execution, tools
- Automated agent scaffolding with /create-agent command
- Complete documentation in .opencode/command/openagents/new-agents/

## OpenCoder Test Suite
- Added 8 comprehensive test cases for OpenCoder agent
- Tests cover: planning, context-loading, delegation, error-handling, implementation
- Includes DEBUG_GUIDE.md and QUICK_TEST_GUIDE.md for developers

## Documentation & Tooling
- Added DEVELOPMENT.md guide for contributors
- Enhanced CONTRIBUTING.md with quick reference and common commands
- Improved event-logger with DEBUG_VERBOSE mode for detailed output
- Added debug scripts: show-test-conversation.sh, run-test-verbose.sh
- Removed deprecated EXTERNAL_PR_GUIDE.md

## Version Bump
- Bumped version to 0.0.3 (patch)

Files changed: 60+ files (21 test updates, 16 new agent templates, 8 new tests, docs, tooling)

* feat(ci): add PR title validation for semantic versioning

Add automated PR title validation to ensure conventional commit format.
This enables proper automatic version bumping when PRs are merged.

Features:
- Validates PR titles against conventional commit patterns
- Shows expected version bump (major/minor/patch)
- Posts helpful comment with examples if validation fails
- Supports all conventional commit types (feat, fix, docs, etc.)
- Supports breaking changes (feat!, fix!)
- Supports pre-release tags ([alpha], [beta], [rc])

When PR titles follow the format, version bumping works correctly:
- feat: → minor bump (0.3.0 → 0.4.0)
- fix: → patch bump (0.3.0 → 0.3.1)
- feat!: → major bump (0.3.0 → 1.0.0)

* fix(ci): improve PR checks workflow with sequential execution

Reorganize PR checks to run in logical sequence and skip unnecessary checks:

Changes:
- Run PR title validation first (fast, always required)
- Detect changed files to determine which checks to run
- Only run build checks if evals/ files changed
- Add comprehensive summary job showing all check results
- Prevent cache errors when evals files not changed

Benefits:
- Faster PR checks (skip unnecessary builds)
- Clear sequential execution (title → changes → build)
- Better error messages and summaries
- Reduced CI/CD costs

* fix(ci): resolve PR #42 issues and refactor prompt architecture

## Fixed PR #42 Issues

1. **Prompt Validation** - Refactored to new architecture where agent files are canonical defaults
2. **Package-lock.json** - Fixed workflow to use root package-lock (npm workspaces)
3. **Git Merge Base** - Added fetch-depth: 0 to PR checks workflow

## New Prompt Architecture

**Before:**
- Agent files had to match .opencode/prompts/<agent>/default.md
- Caused validation failures when updating prompts
- Redundant duplication

**After:**
- Agent files (.opencode/agent/*.md) = Canonical defaults (source of truth)
- Prompt variants (.opencode/prompts/<agent>/<model>.md) = Model-specific optimizations
- No more default.md files needed

## Changes

### Workflows
- pr-checks.yml: Added fetch-depth: 0 for full git history
- validate-test-suites.yml: Fixed package-lock path for npm workspaces
- validate-registry.yml: Updated validation messaging

### Scripts
- validate-pr.sh: Validates prompt library structure (rejects default.md files)
- use-prompt.sh: Handles 'default' as agent file, variants as model-specific
- test-prompt.sh: Tests default variant correctly, saves results to prompts/results/

### Documentation
- .opencode/prompts/README.md: Updated architecture explanation
- docs/contributing/CONTRIBUTING.md: Updated prompt workflow
- scripts/development/demo.sh: Updated structure display

## Results

- 12 files changed, 231 insertions(+), 811 deletions(-)
- Net reduction: 580 lines (cleaner codebase)
- All validations passing
- Results directory structure preserved
- Backwards compatible with existing test results
2025-12-09 13:26:52 +00:00

106 lines
3.3 KiB
Bash
Executable file

#!/bin/bash
# Validate prompt library structure
# Agent files in .opencode/agent/ are the canonical defaults
# Prompts in .opencode/prompts/ are model-specific variants
set -e
AGENT_DIR=".opencode/agent"
PROMPTS_DIR=".opencode/prompts"
FAILED=0
WARNINGS=0
# Colors
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
BLUE='\033[0;34m'
NC='\033[0m' # No Color
echo "🔍 Validating prompt library structure..."
echo ""
# Check if prompts directory exists
if [ ! -d "$PROMPTS_DIR" ]; then
echo -e "${YELLOW}⚠️ Prompts library not yet set up${NC}"
echo " This is expected if the prompt library system hasn't been implemented yet."
echo " Skipping validation."
exit 0
fi
# Validate structure: agent files are canonical, prompts are variants
echo -e "${BLUE}Architecture:${NC}"
echo " • Agent files (.opencode/agent/*.md) = Canonical defaults"
echo " • Prompt variants (.opencode/prompts/<agent>/<model>.md) = Model-specific optimizations"
echo ""
# Find all agent markdown files (excluding subagents)
for agent_file in "$AGENT_DIR"/*.md; do
# Skip if no files found
[ -e "$agent_file" ] || continue
agent_name=$(basename "$agent_file" .md)
prompts_subdir="$PROMPTS_DIR/$agent_name"
# Check if prompts directory exists for this agent
if [ ! -d "$prompts_subdir" ]; then
echo -e "${YELLOW}⚠️ No prompt variants for $agent_name${NC}"
echo " Agent file: $agent_file (canonical default)"
echo " Variants directory: $prompts_subdir (not found)"
echo " This is OK - variants are optional."
echo ""
WARNINGS=$((WARNINGS + 1))
continue
fi
# Check for default.md (should NOT exist in new architecture)
if [ -f "$prompts_subdir/default.md" ]; then
echo -e "${RED}❌ Found default.md for $agent_name${NC}"
echo " Location: $prompts_subdir/default.md"
echo " This file should not exist in the new architecture."
echo ""
echo " To fix:"
echo " rm $prompts_subdir/default.md"
echo ""
echo " The agent file is now the canonical default:"
echo " $agent_file"
echo ""
FAILED=$((FAILED + 1))
else
# Count variants
variant_count=$(find "$prompts_subdir" -maxdepth 1 -name "*.md" -not -name "README.md" -not -name "TEMPLATE.md" | wc -l | tr -d ' ')
if [ "$variant_count" -gt 0 ]; then
echo -e "${GREEN}✅ $agent_name${NC}"
echo " Default: $agent_file"
echo " Variants: $variant_count model-specific optimization(s)"
else
echo -e "${GREEN}✅ $agent_name${NC}"
echo " Default: $agent_file"
echo " Variants: none (using default for all models)"
fi
fi
done
echo ""
# Summary
if [ $FAILED -eq 0 ] && [ $WARNINGS -eq 0 ]; then
echo -e "${GREEN}✅ Prompt library structure is valid${NC}"
exit 0
elif [ $FAILED -eq 0 ]; then
echo -e "${YELLOW}⚠️ Validation passed with $WARNINGS warning(s)${NC}"
echo " Some agents don't have variant directories yet - this is expected."
exit 0
else
echo -e "${RED}❌ PR validation failed - $FAILED issue(s) found${NC}"
echo ""
echo "The new prompt architecture:"
echo " • Agent files (.opencode/agent/*.md) are the canonical defaults"
echo " • Prompt variants (.opencode/prompts/<agent>/<model>.md) are model-specific"
echo " • default.md files should NOT exist"
echo ""
echo "See docs/contributing/CONTRIBUTING.md for details"
echo ""
exit 1
fi