OpenWriterControle/evals/framework/scripts/debug/show-test-conversation.sh
Darren Hinde 784ffadf92
chore: verify and stabilize main branch (#42)
* chore: update package-lock.json after npm install

* chore: remove unused ajv-cli, keep vitest at stable 1.6.1

- Removed ajv-cli (unused dev dependency, had high severity vulnerability)
- Kept vitest at 1.6.1 (stable, upgrading to v4 requires extensive refactoring)
- Reduced vulnerabilities from 6 to 4 (all moderate, dev-only)
- All remaining vulnerabilities are in esbuild/vite (dev server only, no production impact)
- Build and tests verified working

* feat(evals): add execution-balance validation to 21 OpenAgent tests and enhance agent system

Major improvements to evaluation framework and agent capabilities:

## Execution Balance Evaluator Integration
- Added execution-balance rule to 21 OpenAgent tests (2.7% → 31.5% coverage)
- Validates "read before execute" pattern across critical rules, workflows, and edge cases
- Tests now check: read operations before execution tools, healthy read/exec ratio (≥1.0)
- Coverage includes: approval-gate, context-loading, stop-on-failure, report-first tests

## Agent System Enhancements
- Enhanced OpenCoder agent with critical rules enforcement (approval gates, context loading)
- Added structured delegation rules and workflow phases
- Improved error handling with REPORT→PROPOSE→REQUEST→FIX pattern

## New Agent Creation System
- Added research-backed agent creation templates and guides
- Includes 8 test templates covering planning, context, incremental execution, tools
- Automated agent scaffolding with /create-agent command
- Complete documentation in .opencode/command/openagents/new-agents/

## OpenCoder Test Suite
- Added 8 comprehensive test cases for OpenCoder agent
- Tests cover: planning, context-loading, delegation, error-handling, implementation
- Includes DEBUG_GUIDE.md and QUICK_TEST_GUIDE.md for developers

## Documentation & Tooling
- Added DEVELOPMENT.md guide for contributors
- Enhanced CONTRIBUTING.md with quick reference and common commands
- Improved event-logger with DEBUG_VERBOSE mode for detailed output
- Added debug scripts: show-test-conversation.sh, run-test-verbose.sh
- Removed deprecated EXTERNAL_PR_GUIDE.md

## Version Bump
- Bumped version to 0.0.3 (patch)

Files changed: 60+ files (21 test updates, 16 new agent templates, 8 new tests, docs, tooling)

* feat(ci): add PR title validation for semantic versioning

Add automated PR title validation to ensure conventional commit format.
This enables proper automatic version bumping when PRs are merged.

Features:
- Validates PR titles against conventional commit patterns
- Shows expected version bump (major/minor/patch)
- Posts helpful comment with examples if validation fails
- Supports all conventional commit types (feat, fix, docs, etc.)
- Supports breaking changes (feat!, fix!)
- Supports pre-release tags ([alpha], [beta], [rc])

When PR titles follow the format, version bumping works correctly:
- feat: → minor bump (0.3.0 → 0.4.0)
- fix: → patch bump (0.3.0 → 0.3.1)
- feat!: → major bump (0.3.0 → 1.0.0)

* fix(ci): improve PR checks workflow with sequential execution

Reorganize PR checks to run in logical sequence and skip unnecessary checks:

Changes:
- Run PR title validation first (fast, always required)
- Detect changed files to determine which checks to run
- Only run build checks if evals/ files changed
- Add comprehensive summary job showing all check results
- Prevent cache errors when evals files not changed

Benefits:
- Faster PR checks (skip unnecessary builds)
- Clear sequential execution (title → changes → build)
- Better error messages and summaries
- Reduced CI/CD costs

* fix(ci): resolve PR #42 issues and refactor prompt architecture

## Fixed PR #42 Issues

1. **Prompt Validation** - Refactored to new architecture where agent files are canonical defaults
2. **Package-lock.json** - Fixed workflow to use root package-lock (npm workspaces)
3. **Git Merge Base** - Added fetch-depth: 0 to PR checks workflow

## New Prompt Architecture

**Before:**
- Agent files had to match .opencode/prompts/<agent>/default.md
- Caused validation failures when updating prompts
- Redundant duplication

**After:**
- Agent files (.opencode/agent/*.md) = Canonical defaults (source of truth)
- Prompt variants (.opencode/prompts/<agent>/<model>.md) = Model-specific optimizations
- No more default.md files needed

## Changes

### Workflows
- pr-checks.yml: Added fetch-depth: 0 for full git history
- validate-test-suites.yml: Fixed package-lock path for npm workspaces
- validate-registry.yml: Updated validation messaging

### Scripts
- validate-pr.sh: Validates prompt library structure (rejects default.md files)
- use-prompt.sh: Handles 'default' as agent file, variants as model-specific
- test-prompt.sh: Tests default variant correctly, saves results to prompts/results/

### Documentation
- .opencode/prompts/README.md: Updated architecture explanation
- docs/contributing/CONTRIBUTING.md: Updated prompt workflow
- scripts/development/demo.sh: Updated structure display

## Results

- 12 files changed, 231 insertions(+), 811 deletions(-)
- Net reduction: 580 lines (cleaner codebase)
- All validations passing
- Results directory structure preserved
- Backwards compatible with existing test results
2025-12-09 13:26:52 +00:00

104 lines
3.9 KiB
Bash
Executable file

#!/bin/bash
# Show full conversation from a test session
# Usage: ./show-test-conversation.sh <session_id>
SESSION_ID=$1
if [ -z "$SESSION_ID" ]; then
echo "Usage: $0 <session_id>"
echo ""
echo "Get session ID from test output (look for 'Session created: ses_...')"
exit 1
fi
SESSION_DIR="$HOME/.local/share/opencode/storage/message/$SESSION_ID"
PART_DIR="$HOME/.local/share/opencode/storage/part"
if [ ! -d "$SESSION_DIR" ]; then
echo "Session not found: $SESSION_ID"
exit 1
fi
echo "========================================================================"
echo "SESSION: $SESSION_ID"
echo "========================================================================"
echo ""
# Process messages in order
for msg_file in "$SESSION_DIR"/*.json; do
if [ -f "$msg_file" ]; then
MSG_ID=$(cat "$msg_file" | jq -r '.id')
ROLE=$(cat "$msg_file" | jq -r '.role')
SUMMARY=$(cat "$msg_file" | jq -r '.summary.body // empty')
if [ "$ROLE" = "user" ]; then
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
echo "👤 USER PROMPT"
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
# Get actual user prompt from parts (not summary which is auto-generated)
if [ -d "$PART_DIR/$MSG_ID" ]; then
for part_file in "$PART_DIR/$MSG_ID"/*.json; do
if [ -f "$part_file" ]; then
PART_TYPE=$(cat "$part_file" | jq -r '.type')
if [ "$PART_TYPE" = "text" ]; then
TEXT=$(cat "$part_file" | jq -r '.text // empty')
if [ -n "$TEXT" ]; then
echo "$TEXT"
fi
fi
fi
done
else
# Fallback to summary if no parts (shouldn't happen)
if [ -n "$SUMMARY" ]; then
echo "$SUMMARY"
fi
fi
echo ""
elif [ "$ROLE" = "assistant" ]; then
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
echo "🤖 ASSISTANT"
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
# Read parts from the part directory
if [ -d "$PART_DIR/$MSG_ID" ]; then
for part_file in "$PART_DIR/$MSG_ID"/*.json; do
if [ -f "$part_file" ]; then
PART_TYPE=$(cat "$part_file" | jq -r '.type')
if [ "$PART_TYPE" = "text" ]; then
TEXT=$(cat "$part_file" | jq -r '.text // empty')
if [ -n "$TEXT" ]; then
echo "$TEXT"
echo ""
fi
elif [ "$PART_TYPE" = "tool" ]; then
TOOL=$(cat "$part_file" | jq -r '.tool')
INPUT=$(cat "$part_file" | jq -c '.input // {}')
echo "🔧 TOOL CALL: $TOOL"
echo " Input: $INPUT"
echo ""
elif [ "$PART_TYPE" = "tool_result" ]; then
RESULT=$(cat "$part_file" | jq -r '.result // empty')
if [ -n "$RESULT" ]; then
# Show first 500 chars of result
echo "📊 TOOL RESULT:"
echo "$RESULT" | head -c 500
if [ ${#RESULT} -gt 500 ]; then
echo "..."
fi
echo ""
fi
fi
fi
done
fi
echo ""
fi
fi
done
echo "========================================================================"