f773b290ce
Documentation: - Add ARCHITECTURE.md with comprehensive system review - Add GETTING_STARTED.md for new users - Add DOCUMENTATION_CLEANUP.md tracking cleanup changes - Add SCRIPTS_ORGANIZATION.md documenting script reorganization - Remove outdated docs (TESTING_CONFIDENCE.md, TEST_REVIEW.md, SESSION_STORAGE_FIX.md) - Update README files with current information Script Organization: - Move 12 scripts from framework root to organized directories: - scripts/debug/ (4 files): Session debugging tools - scripts/test/ (6 files): Test execution scripts - scripts/utils/ (2 files): Utility scripts - Add comprehensive scripts/README.md with usage examples Test Infrastructure: - Add 5 new context loading tests for openagent - Add result-saver.ts for test result persistence - Add interactive results dashboard (index.html) - Add error-handling.test.ts for framework testing - Add result-saver.test.ts with comprehensive unit tests Evaluator Improvements: - Enhance context-loading-evaluator with better detection - Update run-sdk-tests.ts with improved result handling Agent Configuration: - Add .opencode/agent/AGENT.md for agent definition Test Utilities: - Add test-agent-manual.mjs for manual testing - Update test_tmp/README.md with usage guidelines This commit consolidates multiple improvements: - Better organization (scripts, docs) - Improved testing infrastructure - Comprehensive documentation - Cleaner codebase structure
44 lines
1.1 KiB
YAML
44 lines
1.1 KiB
YAML
id: ctx-simple-testing-approach
|
|
name: "Context Loading: Testing Approach"
|
|
description: |
|
|
Simple test: Ask about testing strategy and verify agent loads context file before responding.
|
|
|
|
Expected behavior:
|
|
- Agent should read tests.md or testing.md BEFORE responding
|
|
- Agent should reference project-specific testing standards
|
|
|
|
Files created: None (read-only test)
|
|
Working directory: evals/test_tmp/
|
|
|
|
category: developer
|
|
agent: openagent
|
|
model: anthropic/claude-sonnet-4-5
|
|
|
|
# Single prompt test
|
|
prompt: "What's our testing strategy for this project?"
|
|
|
|
# Expected behavior
|
|
behavior:
|
|
mustUseAnyOf:
|
|
- [read] # Must use read tool
|
|
requiresContext: true # Must load context before responding
|
|
minToolCalls: 1 # At least one read
|
|
|
|
# Expected violations
|
|
expectedViolations:
|
|
- rule: context-loading
|
|
shouldViolate: false
|
|
severity: error
|
|
description: Must load tests.md or testing.md before responding
|
|
|
|
# Approval strategy
|
|
approvalStrategy:
|
|
type: auto-approve
|
|
|
|
timeout: 60000
|
|
|
|
tags:
|
|
- context-loading
|
|
- simple-test
|
|
- read-only
|
|
- testing
|