f773b290ce
Documentation: - Add ARCHITECTURE.md with comprehensive system review - Add GETTING_STARTED.md for new users - Add DOCUMENTATION_CLEANUP.md tracking cleanup changes - Add SCRIPTS_ORGANIZATION.md documenting script reorganization - Remove outdated docs (TESTING_CONFIDENCE.md, TEST_REVIEW.md, SESSION_STORAGE_FIX.md) - Update README files with current information Script Organization: - Move 12 scripts from framework root to organized directories: - scripts/debug/ (4 files): Session debugging tools - scripts/test/ (6 files): Test execution scripts - scripts/utils/ (2 files): Utility scripts - Add comprehensive scripts/README.md with usage examples Test Infrastructure: - Add 5 new context loading tests for openagent - Add result-saver.ts for test result persistence - Add interactive results dashboard (index.html) - Add error-handling.test.ts for framework testing - Add result-saver.test.ts with comprehensive unit tests Evaluator Improvements: - Enhance context-loading-evaluator with better detection - Update run-sdk-tests.ts with improved result handling Agent Configuration: - Add .opencode/agent/AGENT.md for agent definition Test Utilities: - Add test-agent-manual.mjs for manual testing - Update test_tmp/README.md with usage guidelines This commit consolidates multiple improvements: - Better organization (scripts, docs) - Improved testing infrastructure - Comprehensive documentation - Cleaner codebase structure
44 lines
1.1 KiB
YAML
44 lines
1.1 KiB
YAML
id: ctx-simple-documentation-format
|
|
name: "Context Loading: Documentation Format"
|
|
description: |
|
|
Simple test: Ask about documentation format and verify agent loads context file before responding.
|
|
|
|
Expected behavior:
|
|
- Agent should read docs.md or documentation.md BEFORE responding
|
|
- Agent should reference project-specific documentation standards
|
|
|
|
Files created: None (read-only test)
|
|
Working directory: evals/test_tmp/
|
|
|
|
category: developer
|
|
agent: openagent
|
|
model: anthropic/claude-sonnet-4-5
|
|
|
|
# Single prompt test
|
|
prompt: "What format should I use for documentation in this project?"
|
|
|
|
# Expected behavior
|
|
behavior:
|
|
mustUseAnyOf:
|
|
- [read] # Must use read tool
|
|
requiresContext: true # Must load context before responding
|
|
minToolCalls: 1 # At least one read
|
|
|
|
# Expected violations
|
|
expectedViolations:
|
|
- rule: context-loading
|
|
shouldViolate: false
|
|
severity: error
|
|
description: Must load docs.md or documentation.md before responding
|
|
|
|
# Approval strategy
|
|
approvalStrategy:
|
|
type: auto-approve
|
|
|
|
timeout: 60000
|
|
|
|
tags:
|
|
- context-loading
|
|
- simple-test
|
|
- read-only
|
|
- documentation
|