OpenWriterControle/CONTEXT_SYSTEM_GUIDE.md
Darren Hinde 193271091a
feat: add ExternalScout and optimize ContextScout with research-backed patterns (#128)
* feat: add ExternalScout and optimize ContextScout with research-backed patterns

## New Features

### ExternalScout Subagent (v2.0.0)
- Fetches live, version-specific documentation for external libraries via Context7 API
- Supports 18+ libraries: Drizzle, Better Auth, Next.js, TanStack, Cloudflare Workers, AWS Lambda, Vercel, Shadcn/ui, Radix UI, Tailwind CSS, Zustand, Jotai, Zod, React Hook Form, Vitest, Playwright
- Lazy-loads query patterns for specific libraries only
- Filters and formats results by relevance (Critical → High → Medium)
- Fallback to official docs via webfetch
- 42% token reduction via research-backed optimization
- Added to registry with skill:context7 dependency

### Context7 Skill
- Registered in system for external documentation fetching
- Library registry with 18+ supported libraries
- Query optimization patterns per library
- Navigation guide and README

## Optimizations

### ContextScout v5.1.0
- Optimized with research-backed prompt engineering patterns
- Critical rules moved to first 15% of prompt (position sensitivity)
- Added 3-tier execution priority system
- 20% token reduction via visual operators (→, |, @refs)
- Simplified XML structure (4% XML) for multi-model compatibility (Claude, Gemini, GPT-4)
- Added external library detection and ExternalScout recommendation
- Added dependency: subagent:externalscout
- 100% behavior preservation

### ExternalScout v2.0.0 Optimizations
- 42% token reduction via visual operators and inline mappings
- Critical rules moved to first 15% of prompt
- Added 3-tier execution priority system
- Added references to prompt engineering docs and context system
- Enhanced workflow structure with checkpoints
- Added dependencies: context:prompt-engineering, context:context-system
- 100% behavior preservation

## Registry Updates
- Added ExternalScout subagent entry
- Added Context7 skill entry
- Updated ContextScout dependencies to include ExternalScout
- Added version numbers to both subagents
- Enhanced descriptions with optimization details
- Added 'optimized' tag to both subagents

## Files Added
- .opencode/agent/subagents/core/externalscout.md
- .opencode/skill/context7/README.md
- .opencode/skill/context7/library-registry.md
- .opencode/skill/context7/navigation.md
- .opencode/agent/subagents/core/contextscout-v4-backup.md (backup)
- .opencode/agent/subagents/core/externalscout-v1-backup.md (backup)

## Files Modified
- .opencode/agent/subagents/core/contextscout.md (v4.0.0 → v5.1.0)
- registry.json (added ExternalScout, Context7, updated ContextScout)
- CHANGELOG.md (documented all changes)
- VERSION (0.5.5 → 0.6.0)
- package.json (0.5.5 → 0.6.0)

## Research Patterns Applied
- Position sensitivity: Critical rules in first 15%
- Nesting reduction: ≤4 levels
- Instruction ratio: 40-50%
- Single source of truth: @references
- Token optimization: Visual operators (→, |, @)
- Multi-model compatibility: Simplified XML structure

## Breaking Changes
None - 100% backward compatible

## Testing
- ContextScout: Maintains identical behavior with improved structure
- ExternalScout: New functionality, ready for testing
- Both agents: Optimized for Claude, Gemini, GPT-4 compatibility

Closes #[issue-number-if-applicable]

* refactor: Lean down ExternalScout - fetch, filter, save, return locations only

- Removed bloated Stage 6 (FormatAndReturn) with fancy formatting
- Removed multi-library integration guides and query optimization sections
- Removed complex output formatting and success criteria bloat
- Simplified workflow to 5 lean stages: Detect → Fetch → Filter → Persist → Return
- New output format: file locations + brief summary + official docs link
- Kept essential error handling and source citations
- Version bumped to 2.0.0
- Focus: Fast, accurate documentation retrieval without extra bloat

* feat: Add ExternalScout and optimize context system

- Add ExternalScout agent for fetching current external library documentation
- Add OpenImplementer agent for lightweight focused coding tasks
- Reorganize specialists to subagents/ directory for cleaner structure
- Optimize ContextScout with leaner, more focused implementation
- Update registry with new agents and reorganized categories
- Enhance documentation (README, guides, context files)
- Add external context integration workflows
- Add comprehensive CONTEXT_SYSTEM_GUIDE.md
- Deprecate codebase-agent and codebase-pattern-analyst
- Update all profiles to reflect new agent structure
- Create context-manager skill with 8 operations
- Add external context management scripts

BREAKING CHANGES: None (all changes backward compatible)

Fixes: Outdated training data issues for external libraries

* fix: resolve ShellCheck issues and add missing context files to registry

- Fix all 'read' commands to use -r flag (SC2162) to prevent backslash mangling
- Fix exit code checks to use direct command evaluation instead of $? (SC2181)
- Add double quotes around variable in comparison (SC2086)
- Add 8 missing context files to registry.json:
  - external-libraries.md (new workflow for external library handling)
  - visual-development.md
  - development/README.md
  - learning/README.md
  - product/README.md
  - data/README.md
  - mastra-ai/README.md
  - context-bundle-template.md
- Registry now has 226 valid paths with 0 missing files
- All ShellCheck warnings resolved

* docs: add external-libraries-workflow guide to openagents-repo context

- Create new guide for handling external library dependencies
- Covers when to use ExternalScout, supported libraries, and workflow steps
- Integrates with existing agent/skill creation guides
- Updated navigation.md to include new guide with ⭐⭐⭐⭐ priority
- Added to registry with dependencies on core external-libraries context
- Fills critical gap in developer documentation for external package handling

* fix: resolve ShellCheck SC2145 errors in context router

- Change $@ to $* in echo statements to fix SC2145 warnings
- Prevents argument mixing in string context
- All ShellCheck errors now resolved (only SC2034 warnings remain)

* fix: resolve all ShellCheck warnings in manage-external-context.sh

- Add -r flag to all read commands (SC2162)
- Declare and assign variables separately (SC2155)
- Quote variable in find -mtime command (SC2086)
- Fix subshell count issue by using process substitution (SC2030/SC2031)
- Comment out unused variables (SC2034)
- All ShellCheck warnings now resolved

* fix: remove unused variables in context router to pass ShellCheck

- Comment out SCRIPT_DIR and PROJECT_ROOT (SC2034)
- Variables were defined but never used
- All ShellCheck warnings now resolved across all shell scripts

* fix: replace bash registry validation with Python script

- Create fast Python validation script (validate-registry.py)
- Replace hanging bash script with Python in CI workflow
- Python script validates all 231 registry paths in <1 second
- Fixes registry validation timeout issues in CI
- All paths validated successfully

* chore: trigger CI re-run for workflow updates

* Revert "fix: replace bash registry validation with Python script"

This reverts commit 03843935d075f1350afb721b5235e9406005810f.

* fix: remove verbose flag and add timeout to registry validation

- Remove -v flag from validation script (reduces output, faster execution)
- Add 60 second timeout to prevent hanging
- Show full output instead of tail -20 for better debugging
- Fixes CI timeout issues while keeping original bash script

* feat: add TypeScript registry validator with full skill support

- Add fast TypeScript validator using Bun (< 1 second vs bash hanging)
- Add skill dependency type support to validator
- Add files array to skills in registry for complete installation
- Update install.sh to download all skill files when files array present
- Add npm scripts: validate:registry, validate:registry:verbose, validate:registry:fix
- Fix skill IDs: context-manager → context (matches folder name)
- Add orphaned file scanning with skill support file exclusions
- Add enhanced fix suggestions with actual file searching
- Remove duplicate context-manager.md file
- Add comprehensive README for registry validation scripts

Skills now install completely with all supporting files:
- SKILL.md (main definition)
- router.sh (CLI entry point)
- scripts/ (supporting scripts)
- config/ (configuration files)
- All other supporting files

Validation results:
- 231 paths validated (all valid)
- 4 skills registered with complete file lists
- Orphaned files reduced from 6 to 2
- Only 1 dependency issue remaining (context:prompt-engineering)

* fix: use TypeScript validator in CI instead of hanging bash script

- Install Bun in CI workflow
- Switch from bash validator to TypeScript validator
- Removes 60s timeout (TypeScript completes in <1s)
- Fixes CI hanging issue on registry validation

* chore: retrigger CI checks

Empty commit to retrigger CI after workflow fix in PR #129

* fix: use official Bun setup action in CI

- Replace manual Bun installation with oven-sh/setup-bun@v2
- Fixes PATH issue where bun wasn't available in subsequent steps
- Ensures Bun is properly installed before running TypeScript validator

* chore: retrigger CI after Bun setup fix in PR #131

Empty commit to retrigger CI once PR #131 is merged to main

* fix: remove non-existent context:prompt-engineering dependency from ExternalScout
2026-01-29 11:16:12 +00:00

16 KiB

The Context System: A Complete Guide

What Is Context?

Context is your project's coding standards and patterns stored as markdown files. It tells agents:

  • How you write code (naming conventions, architecture)
  • What libraries you use (React, Next.js, Tailwind, etc.)
  • Your security requirements
  • Your design system
  • Your project-specific patterns

Think of it as a style guide for AI agents.


Why Context Matters

Without Context

You: "Create a React component"
Agent: Creates component in its own style
Result: Doesn't match your project

With Context

You: "Create a React component"
Agent: Loads your React pattern from context
Agent: Creates component matching your style
Result: Perfectly matches your project

How Context Works

The Flow

Your Request
    ↓
Agent receives request
    ↓
ContextScout discovers relevant context files
    ↓
Agent loads context files
    ↓
Agent follows patterns from context
    ↓
Code matches your standards automatically

Example: Creating a React Component

Step 1: You ask for a component

"Create a user profile component"

Step 2: ContextScout discovers

  • core/standards/code-quality.md (modular patterns)
  • ui/web/react-patterns.md (React conventions)
  • project/project-context.md (YOUR patterns)

Step 3: Agent loads context

# From project-context.md

## React Component Pattern

All components should:
- Use TypeScript with strict mode
- Export named component
- Include JSDoc comments
- Use React.FC type

export const UserProfile: React.FC<Props> = ({ user }) => {
  return <div>{user.name}</div>;
};

Step 4: Agent creates component

/**
 * UserProfile - Displays user information
 */
export const UserProfile: React.FC<UserProfileProps> = ({ user }) => {
  return (
    <div className="user-profile">
      <h2>{user.name}</h2>
      <p>{user.email}</p>
    </div>
  );
};

Result: Component matches your patterns automatically!


Context Directory Structure

.opencode/context/
├── core/                           # Universal standards
│   ├── standards/
│   │   ├── code-quality.md        # Modular, functional patterns
│   │   ├── security-patterns.md   # Security best practices
│   │   ├── test-coverage.md       # Testing standards
│   │   └── documentation.md       # Documentation patterns
│   ├── workflows/
│   │   ├── design-iteration.md    # 4-stage UI design
│   │   ├── task-delegation.md     # Task delegation patterns
│   │   ├── external-libraries.md  # Library integration
│   │   └── code-review.md         # Code review process
│   └── task-management/
│       └── standards/
│           └── task-schema.md     # Task JSON schema
│
├── ui/                             # Design & UX
│   └── web/
│       ├── ui-styling-standards.md    # Tailwind + Flowbite
│       ├── animation-patterns.md      # Micro-interactions
│       ├── react-patterns.md          # React conventions
│       └── design-systems.md          # Design system principles
│
├── development/                    # Language-specific
│   ├── backend-navigation.md
│   ├── ui-navigation.md
│   └── [language-specific patterns]
│
└── project/                        # YOUR custom patterns
    └── project-context.md          # Edit this file!

What's Included in Core Context

1. Code Quality Standards (core/standards/code-quality.md)

Teaches agents:

  • Modular design principles
  • Functional programming patterns
  • Pure functions and immutability
  • Composition over inheritance
  • Naming conventions

Example:

// ✅ Pure function (from context)
const add = (a, b) => a + b;

// ❌ Impure (agents avoid this)
let total = 0;
const addToTotal = (value) => { total += value; };

2. Security Patterns (core/standards/security-patterns.md)

Teaches agents:

  • Input validation
  • Authentication checks
  • Authorization patterns
  • Secure error handling
  • Logging best practices

3. Design Iteration (core/workflows/design-iteration.md)

Teaches agents:

  • 4-stage UI design workflow
  • Layout → Theme → Animation → Implementation
  • Approval gates at each stage
  • Design versioning strategy

4. External Libraries (core/workflows/external-libraries.md)

Teaches agents:

  • How to integrate external libraries
  • Common patterns for popular libraries
  • Configuration best practices
  • Dependency management

5. UI Styling Standards (ui/web/ui-styling-standards.md)

Teaches agents:

  • Tailwind CSS conventions
  • Flowbite component usage
  • Color palette standards
  • Responsive design patterns

6. React Patterns (ui/web/react-patterns.md)

Teaches agents:

  • Modern React hooks
  • Component composition
  • State management patterns
  • Performance optimization

Adding Your Own Patterns

Step 1: Edit Your Project Context

# Open your project context file
nano ~/.opencode/context/project/project-context.md

Step 2: Add Your Patterns

# Your Project Patterns

## API Endpoint Pattern

All API endpoints should follow this pattern:

```typescript
export async function POST(request: Request) {
  try {
    // 1. Parse and validate input
    const body = await request.json();
    if (!body.email) {
      return Response.json({ error: 'Email required' }, { status: 400 });
    }

    // 2. Check authentication
    const user = await auth.verify(request);
    if (!user) {
      return Response.json({ error: 'Unauthorized' }, { status: 401 });
    }

    // 3. Process request
    const result = await processRequest(body);

    // 4. Return response
    return Response.json({ success: true, data: result });
  } catch (error) {
    console.error('POST error:', error);
    return Response.json({ error: error.message }, { status: 500 });
  }
}

React Component Pattern

All React components should:

  • Use TypeScript with strict mode
  • Export named component
  • Include JSDoc comments
  • Use React.FC type
  • Include prop validation
interface UserCardProps {
  userId: string;
  onSelect?: (id: string) => void;
}

/**
 * UserCard - Displays user information in a card
 * @param props - Component props
 */
export const UserCard: React.FC<UserCardProps> = ({ 
  userId, 
  onSelect 
}) => {
  const [user, setUser] = React.useState(null);

  React.useEffect(() => {
    fetchUser(userId).then(setUser);
  }, [userId]);

  return (
    <div className="user-card">
      <h3>{user?.name}</h3>
      <button onClick={() => onSelect?.(userId)}>
        Select
      </button>
    </div>
  );
};

Database Query Pattern

All database queries should:

  • Use parameterized queries
  • Include error handling
  • Log important operations
  • Return typed results
export async function getUserById(id: string) {
  try {
    const user = await db
      .select()
      .from(users)
      .where(eq(users.id, id))
      .limit(1);
    
    if (!user.length) {
      throw new Error('User not found');
    }

    return user[0];
  } catch (error) {
    console.error('getUserById error:', error);
    throw error;
  }
}

Security Checklist

All endpoints must:

  • Validate user authentication
  • Check user permissions
  • Sanitize and validate inputs
  • Use parameterized queries
  • Implement rate limiting
  • Log security events
  • Handle errors securely
  • Use HTTPS only

Naming Conventions

  • Files: lowercase-with-dashes.ts
  • Functions: verbPhrases (getUser, validateEmail)
  • Constants: UPPER_CASE
  • Types: PascalCase
  • Variables: camelCase
  • Predicates: isValid, hasPermission, canAccess

Folder Structure

src/
├── components/          # React components
│   ├── common/         # Reusable components
│   ├── features/       # Feature-specific components
│   └── layouts/        # Layout components
├── lib/                # Utilities and helpers
├── api/                # API routes
├── types/              # TypeScript types
├── styles/             # Global styles
└── hooks/              # Custom React hooks

### Step 3: Agents Use Your Patterns

When you ask an agent to create code, it will:
1. Load your project context
2. Follow your patterns
3. Create code matching your style

---

## Real-World Examples

### Example 1: Adding a Component Pattern

**Your project context**:
```markdown
## Button Component Pattern

All buttons should:
- Use Tailwind classes
- Include loading state
- Support disabled state
- Include accessibility attributes

export const Button: React.FC<ButtonProps> = ({
  children,
  isLoading,
  disabled,
  ...props
}) => (
  <button
    disabled={disabled || isLoading}
    className="px-4 py-2 bg-blue-500 text-white rounded hover:bg-blue-600 disabled:opacity-50"
    aria-busy={isLoading}
    {...props}
  >
    {isLoading ? 'Loading...' : children}
  </button>
);

You ask:

"Create a submit button for the login form"

Agent creates:

export const SubmitButton: React.FC<SubmitButtonProps> = ({
  isLoading,
  disabled,
}) => (
  <Button
    type="submit"
    isLoading={isLoading}
    disabled={disabled}
  >
    Sign In
  </Button>
);

✅ Automatically follows your button pattern!

Example 2: Adding an API Pattern

Your project context:

## API Response Pattern

All API responses should:
- Return { success, data, error }
- Include proper status codes
- Log all requests
- Validate input

export async function POST(request: Request) {
  try {
    const body = await request.json();
    
    // Validate
    if (!body.email) {
      return Response.json(
        { success: false, error: 'Email required' },
        { status: 400 }
      );
    }

    // Process
    const result = await process(body);

    // Return
    return Response.json({ success: true, data: result });
  } catch (error) {
    return Response.json(
      { success: false, error: error.message },
      { status: 500 }
    );
  }
}

You ask:

"Create an API endpoint to create a new user"

Agent creates:

export async function POST(request: Request) {
  try {
    const body = await request.json();
    
    // Validate
    if (!body.email || !body.password) {
      return Response.json(
        { success: false, error: 'Email and password required' },
        { status: 400 }
      );
    }

    // Process
    const user = await createUser(body);

    // Return
    return Response.json({ success: true, data: user });
  } catch (error) {
    return Response.json(
      { success: false, error: error.message },
      { status: 500 }
    );
  }
}

✅ Automatically follows your API pattern!

Example 3: Adding a Security Pattern

Your project context:

## Security Requirements

All endpoints must:
- Verify user authentication
- Check user permissions
- Validate all inputs
- Use parameterized queries
- Log security events

Example:
```typescript
export async function POST(request: Request) {
  // 1. Verify authentication
  const user = await auth.verify(request);
  if (!user) {
    return Response.json({ error: 'Unauthorized' }, { status: 401 });
  }

  // 2. Check permissions
  if (!user.isAdmin) {
    return Response.json({ error: 'Forbidden' }, { status: 403 });
  }

  // 3. Validate input
  const body = await request.json();
  if (!validateInput(body)) {
    return Response.json({ error: 'Invalid input' }, { status: 400 });
  }

  // 4. Log security event
  console.log(`Admin action by ${user.id}: create-user`);

  // 5. Process
  const result = await process(body);
  return Response.json({ success: true, data: result });
}

**You ask**:

"Create an admin endpoint to delete a user"


**Agent creates**:
```typescript
export async function DELETE(request: Request) {
  // 1. Verify authentication
  const user = await auth.verify(request);
  if (!user) {
    return Response.json({ error: 'Unauthorized' }, { status: 401 });
  }

  // 2. Check permissions
  if (!user.isAdmin) {
    return Response.json({ error: 'Forbidden' }, { status: 403 });
  }

  // 3. Validate input
  const { userId } = await request.json();
  if (!userId) {
    return Response.json({ error: 'User ID required' }, { status: 400 });
  }

  // 4. Log security event
  console.log(`Admin action by ${user.id}: delete-user ${userId}`);

  // 5. Process
  await deleteUser(userId);
  return Response.json({ success: true });
}

✅ Automatically includes all security checks!


Context Best Practices

1. Keep It Organized

# Your Project Patterns

## Components
[Component patterns]

## API Endpoints
[API patterns]

## Database
[Database patterns]

## Security
[Security requirements]

## Naming
[Naming conventions]

## Folder Structure
[Folder structure]

2. Include Real Examples

## Good ✅
export const Button: React.FC<Props> = ({ ... }) => {
  // Real example from your project
};

## Bad ❌
export const Button = (props) => {
  // Generic example
};

3. Be Specific

## Good ✅
All API endpoints must:
- Validate input with Zod
- Return { success, data, error }
- Log with console.error for errors
- Use 400 for validation, 401 for auth, 500 for server

## Bad ❌
All API endpoints should be good

4. Update Regularly

When you change patterns:

  1. Update project-context.md
  2. Agents will use new patterns immediately
  3. No need to restart anything

Troubleshooting

Problem: Agent isn't following my patterns

Solution:

  1. Check that project-context.md exists
  2. Verify the pattern is clearly written
  3. Include a real example
  4. Ask agent to "follow project patterns"

Problem: Agent uses old patterns

Solution:

  1. Update project-context.md
  2. Restart the agent
  3. Ask agent to "load latest context"

Problem: Pattern is too complex

Solution:

  1. Break into smaller patterns
  2. Include step-by-step example
  3. Add comments explaining why

Problem: Agent creates code that doesn't match

Solution:

  1. Check if pattern is in project-context.md
  2. Verify pattern is specific enough
  3. Ask agent to "review against project patterns"
  4. Provide feedback to refine pattern

Advanced: Context Hierarchy

Agents load context in this order:

  1. Core Standards (universal patterns)

    • core/standards/code-quality.md
    • core/standards/security-patterns.md
  2. Workflows (how to do things)

    • core/workflows/design-iteration.md
    • core/workflows/external-libraries.md
  3. Domain-Specific (language/framework)

    • development/[language]/patterns.md
    • ui/web/react-patterns.md
  4. Project-Specific (YOUR patterns) ← Most important!

    • project/project-context.md

Project context overrides everything else!


Summary

Context is your secret weapon for AI-assisted development:

✅ Automatic pattern following - No manual configuration ✅ Consistent code - All code matches your style ✅ Team alignment - Everyone follows same patterns ✅ Easy updates - Change patterns once, agents use everywhere ✅ Living documentation - Patterns stay in sync with code

Get started:

  1. Edit ~/.opencode/context/project/project-context.md
  2. Add your patterns with examples
  3. Ask agents to create code
  4. Watch them follow your patterns automatically!

Next Steps

  1. Review your current coding patterns
  2. Document them in project-context.md
  3. Include real examples from your project
  4. Test by asking agents to create code
  5. Refine patterns based on results

Your agents will become better and better as your context improves!