OpenWriterControle/.opencode/context/core/workflows/external-libraries.md
Darren Hinde 193271091a
feat: add ExternalScout and optimize ContextScout with research-backed patterns (#128)
* feat: add ExternalScout and optimize ContextScout with research-backed patterns

## New Features

### ExternalScout Subagent (v2.0.0)
- Fetches live, version-specific documentation for external libraries via Context7 API
- Supports 18+ libraries: Drizzle, Better Auth, Next.js, TanStack, Cloudflare Workers, AWS Lambda, Vercel, Shadcn/ui, Radix UI, Tailwind CSS, Zustand, Jotai, Zod, React Hook Form, Vitest, Playwright
- Lazy-loads query patterns for specific libraries only
- Filters and formats results by relevance (Critical → High → Medium)
- Fallback to official docs via webfetch
- 42% token reduction via research-backed optimization
- Added to registry with skill:context7 dependency

### Context7 Skill
- Registered in system for external documentation fetching
- Library registry with 18+ supported libraries
- Query optimization patterns per library
- Navigation guide and README

## Optimizations

### ContextScout v5.1.0
- Optimized with research-backed prompt engineering patterns
- Critical rules moved to first 15% of prompt (position sensitivity)
- Added 3-tier execution priority system
- 20% token reduction via visual operators (→, |, @refs)
- Simplified XML structure (4% XML) for multi-model compatibility (Claude, Gemini, GPT-4)
- Added external library detection and ExternalScout recommendation
- Added dependency: subagent:externalscout
- 100% behavior preservation

### ExternalScout v2.0.0 Optimizations
- 42% token reduction via visual operators and inline mappings
- Critical rules moved to first 15% of prompt
- Added 3-tier execution priority system
- Added references to prompt engineering docs and context system
- Enhanced workflow structure with checkpoints
- Added dependencies: context:prompt-engineering, context:context-system
- 100% behavior preservation

## Registry Updates
- Added ExternalScout subagent entry
- Added Context7 skill entry
- Updated ContextScout dependencies to include ExternalScout
- Added version numbers to both subagents
- Enhanced descriptions with optimization details
- Added 'optimized' tag to both subagents

## Files Added
- .opencode/agent/subagents/core/externalscout.md
- .opencode/skill/context7/README.md
- .opencode/skill/context7/library-registry.md
- .opencode/skill/context7/navigation.md
- .opencode/agent/subagents/core/contextscout-v4-backup.md (backup)
- .opencode/agent/subagents/core/externalscout-v1-backup.md (backup)

## Files Modified
- .opencode/agent/subagents/core/contextscout.md (v4.0.0 → v5.1.0)
- registry.json (added ExternalScout, Context7, updated ContextScout)
- CHANGELOG.md (documented all changes)
- VERSION (0.5.5 → 0.6.0)
- package.json (0.5.5 → 0.6.0)

## Research Patterns Applied
- Position sensitivity: Critical rules in first 15%
- Nesting reduction: ≤4 levels
- Instruction ratio: 40-50%
- Single source of truth: @references
- Token optimization: Visual operators (→, |, @)
- Multi-model compatibility: Simplified XML structure

## Breaking Changes
None - 100% backward compatible

## Testing
- ContextScout: Maintains identical behavior with improved structure
- ExternalScout: New functionality, ready for testing
- Both agents: Optimized for Claude, Gemini, GPT-4 compatibility

Closes #[issue-number-if-applicable]

* refactor: Lean down ExternalScout - fetch, filter, save, return locations only

- Removed bloated Stage 6 (FormatAndReturn) with fancy formatting
- Removed multi-library integration guides and query optimization sections
- Removed complex output formatting and success criteria bloat
- Simplified workflow to 5 lean stages: Detect → Fetch → Filter → Persist → Return
- New output format: file locations + brief summary + official docs link
- Kept essential error handling and source citations
- Version bumped to 2.0.0
- Focus: Fast, accurate documentation retrieval without extra bloat

* feat: Add ExternalScout and optimize context system

- Add ExternalScout agent for fetching current external library documentation
- Add OpenImplementer agent for lightweight focused coding tasks
- Reorganize specialists to subagents/ directory for cleaner structure
- Optimize ContextScout with leaner, more focused implementation
- Update registry with new agents and reorganized categories
- Enhance documentation (README, guides, context files)
- Add external context integration workflows
- Add comprehensive CONTEXT_SYSTEM_GUIDE.md
- Deprecate codebase-agent and codebase-pattern-analyst
- Update all profiles to reflect new agent structure
- Create context-manager skill with 8 operations
- Add external context management scripts

BREAKING CHANGES: None (all changes backward compatible)

Fixes: Outdated training data issues for external libraries

* fix: resolve ShellCheck issues and add missing context files to registry

- Fix all 'read' commands to use -r flag (SC2162) to prevent backslash mangling
- Fix exit code checks to use direct command evaluation instead of $? (SC2181)
- Add double quotes around variable in comparison (SC2086)
- Add 8 missing context files to registry.json:
  - external-libraries.md (new workflow for external library handling)
  - visual-development.md
  - development/README.md
  - learning/README.md
  - product/README.md
  - data/README.md
  - mastra-ai/README.md
  - context-bundle-template.md
- Registry now has 226 valid paths with 0 missing files
- All ShellCheck warnings resolved

* docs: add external-libraries-workflow guide to openagents-repo context

- Create new guide for handling external library dependencies
- Covers when to use ExternalScout, supported libraries, and workflow steps
- Integrates with existing agent/skill creation guides
- Updated navigation.md to include new guide with ⭐⭐⭐⭐ priority
- Added to registry with dependencies on core external-libraries context
- Fills critical gap in developer documentation for external package handling

* fix: resolve ShellCheck SC2145 errors in context router

- Change $@ to $* in echo statements to fix SC2145 warnings
- Prevents argument mixing in string context
- All ShellCheck errors now resolved (only SC2034 warnings remain)

* fix: resolve all ShellCheck warnings in manage-external-context.sh

- Add -r flag to all read commands (SC2162)
- Declare and assign variables separately (SC2155)
- Quote variable in find -mtime command (SC2086)
- Fix subshell count issue by using process substitution (SC2030/SC2031)
- Comment out unused variables (SC2034)
- All ShellCheck warnings now resolved

* fix: remove unused variables in context router to pass ShellCheck

- Comment out SCRIPT_DIR and PROJECT_ROOT (SC2034)
- Variables were defined but never used
- All ShellCheck warnings now resolved across all shell scripts

* fix: replace bash registry validation with Python script

- Create fast Python validation script (validate-registry.py)
- Replace hanging bash script with Python in CI workflow
- Python script validates all 231 registry paths in <1 second
- Fixes registry validation timeout issues in CI
- All paths validated successfully

* chore: trigger CI re-run for workflow updates

* Revert "fix: replace bash registry validation with Python script"

This reverts commit 03843935d075f1350afb721b5235e9406005810f.

* fix: remove verbose flag and add timeout to registry validation

- Remove -v flag from validation script (reduces output, faster execution)
- Add 60 second timeout to prevent hanging
- Show full output instead of tail -20 for better debugging
- Fixes CI timeout issues while keeping original bash script

* feat: add TypeScript registry validator with full skill support

- Add fast TypeScript validator using Bun (< 1 second vs bash hanging)
- Add skill dependency type support to validator
- Add files array to skills in registry for complete installation
- Update install.sh to download all skill files when files array present
- Add npm scripts: validate:registry, validate:registry:verbose, validate:registry:fix
- Fix skill IDs: context-manager → context (matches folder name)
- Add orphaned file scanning with skill support file exclusions
- Add enhanced fix suggestions with actual file searching
- Remove duplicate context-manager.md file
- Add comprehensive README for registry validation scripts

Skills now install completely with all supporting files:
- SKILL.md (main definition)
- router.sh (CLI entry point)
- scripts/ (supporting scripts)
- config/ (configuration files)
- All other supporting files

Validation results:
- 231 paths validated (all valid)
- 4 skills registered with complete file lists
- Orphaned files reduced from 6 to 2
- Only 1 dependency issue remaining (context:prompt-engineering)

* fix: use TypeScript validator in CI instead of hanging bash script

- Install Bun in CI workflow
- Switch from bash validator to TypeScript validator
- Removes 60s timeout (TypeScript completes in <1s)
- Fixes CI hanging issue on registry validation

* chore: retrigger CI checks

Empty commit to retrigger CI after workflow fix in PR #129

* fix: use official Bun setup action in CI

- Replace manual Bun installation with oven-sh/setup-bun@v2
- Fixes PATH issue where bun wasn't available in subsequent steps
- Ensures Bun is properly installed before running TypeScript validator

* chore: retrigger CI after Bun setup fix in PR #131

Empty commit to retrigger CI once PR #131 is merged to main

* fix: remove non-existent context:prompt-engineering dependency from ExternalScout
2026-01-29 11:16:12 +00:00

16 KiB

Workflow: External Libraries

Purpose: Fetch current documentation for external packages before implementation

Last Updated: 2026-01-28


Quick Start

Golden Rule: NEVER rely on training data for external libraries → ALWAYS fetch current docs

Process: Detect package → Check install scripts → Use ExternalScout → Implement

When to use ExternalScout (MANDATORY):

  • New builds w/ external packages
  • First-time package setup
  • Package/dependency errors
  • Version upgrades
  • ANY external library work

Core Principle

Training data is OUTDATED for external libraries. ALWAYS fetch current docs using ExternalScout before implementation.

Why:

  • APIs change (new methods, deprecated features)
  • Configuration patterns evolve
  • Breaking changes happen frequently
  • Version-specific features differ

Example:

Training data (2023): Next.js 13 uses pages/ directory
Current (2025): Next.js 15 uses app/ directory (App Router)

Training data = broken code ❌
ExternalScout = working code ✅

Workflow Stages

1. Detect External Package

Triggers: User mentions library | package.json deps | import statements | build errors | first-time setup

Action: Identify which external packages involved


2. Check Install Scripts (First-Time Builds)

Check for:

# Look for install scripts
ls scripts/install/ scripts/setup/ bin/install* setup.sh install.sh

# Check package-specific requirements
grep -r "postinstall\|preinstall" package.json

Read scripts if found:

  • What does it do?
  • Environment variables needed?
  • Prerequisites (database, services)?

Why: Scripts may set up databases, generate files, configure services in specific order


3. Fetch Current Documentation (MANDATORY)

Basic Pattern:

task(
  subagent_type="ExternalScout",
  description="Fetch [Library] docs for [topic]",
  prompt="Fetch current documentation for [Library]: [specific question]
  
  Focus on:
  - Installation and setup steps
  - [Specific feature/API needed]
  - [Integration requirements]
  - Required environment variables
  - Database/service setup
  
  Context: [What you're building]"
)

Real Examples:

Drizzle ORM Setup:

task(
  subagent_type="ExternalScout",
  description="Fetch Drizzle PostgreSQL setup",
  prompt="Fetch Drizzle ORM docs: PostgreSQL setup w/ modular schemas
  
  Focus on: Installation | DB connection | Schema patterns | Migrations | TypeScript config
  Context: Next.js commerce site w/ PostgreSQL"
)

Next.js App Router:

task(
  subagent_type="ExternalScout",
  description="Fetch Next.js App Router docs",
  prompt="Fetch Next.js docs: App Router w/ Server Actions
  
  Focus on: Installation | Directory structure | Server Actions | Data fetching | Route handlers
  Context: Commerce site w/ order processing"
)

Better Auth Integration:

task(
  subagent_type="ExternalScout",
  description="Fetch Better Auth + Next.js integration",
  prompt="Fetch Better Auth docs: Next.js App Router integration w/ Drizzle
  
  Focus on: Installation | App Router setup | Drizzle adapter | Session mgmt | Route protection
  Context: Adding auth to Next.js commerce w/ Drizzle ORM"
)

ExternalScout Returns:

  • Current, version-specific docs
  • Installation steps + required packages
  • Official API references
  • Working code examples
  • Current best practices
  • Integration patterns
  • Setup prerequisites
  • Environment variables
  • Common pitfalls + solutions

4. Verify Compatibility

Check: Version compatibility | Peer dependencies | Breaking changes | Platform requirements

If version mismatch: Note in plan → Request approval for upgrade → Fetch docs for specific version


5. Implement with Current Patterns

Apply docs:

  • Use exact API signatures from docs
  • Follow current best practices
  • Use recommended config patterns
  • Implement error handling as documented

Don't:

  • ❌ Assume API based on training data
  • ❌ Use deprecated patterns
  • ❌ Skip version-specific requirements
  • ❌ Ignore breaking changes

6. Test Integration

Verify: Package installs | Imports work | API calls match docs | Error handling | Integration w/ other packages


Decision Flow: ContextScout + ExternalScout

User Request: "Build Next.js commerce w/ Drizzle"
                    ↓
┌──────────────────────────────────────────────────┐
│ STEP 1: ContextScout                             │
│ → Search internal context (.opencode/context/)  │
│ → Find project standards, patterns, workflows   │
└──────────────────────────────────────────────────┘
                    ↓
         Internal context found?
                    ↓
        ┌───────────┴───────────┐
       YES                      NO
        │                        │
        ↓                        ↓
   Use internal          Is it external library?
   context                       ↓
                        ┌────────┴────────┐
                       YES               NO
                        │                 │
                        ↓                 ↓
              ┌─────────────────┐  Report: No context
              │ STEP 2:         │  available
              │ ExternalScout   │
              │ (MANDATORY)     │
              └─────────────────┘
                        ↓
              Fetch: Next.js docs
                     Drizzle docs
                     Integration patterns
                        ↓
              ┌─────────────────────────┐
              │ STEP 3: Combine         │
              │ Internal: Standards     │
              │ External: Library docs  │
              │ → Implement w/ both     │
              └─────────────────────────┘

When to Use Which:

Scenario ContextScout ExternalScout Both
Project coding standards ✅ ❌ ❌
External library setup ❌ ✅ MANDATORY ❌
Project-specific patterns ✅ ❌ ❌
External API usage ❌ ✅ MANDATORY ❌
Feature w/ external lib ✅ standards ✅ lib docs ✅
Package installation ❌ ✅ MANDATORY ❌
Security patterns ✅ ❌ ❌
External lib integration ✅ project ✅ lib docs ✅

Key Principle: ContextScout + ExternalScout = Complete Context

  • ContextScout: "How we do things in THIS project"
  • ExternalScout: "How to use THIS library (current version)"
  • Combined: "How to use THIS library following OUR standards"

Common Scenarios

Scenario 1: New Build w/ External Packages

Example: Next.js app w/ Drizzle + Better Auth

Process:

  1. Check install scripts: ls scripts/install/ scripts/setup/
  2. Identify packages: Next.js, Drizzle ORM, Better Auth
  3. ExternalScout for each: Installation | Setup | Integration
  4. Check requirements: PostgreSQL? Env vars? Services?
  5. Verify version compatibility
  6. Run install scripts (if exist) OR create setup from docs
  7. Implement following current docs
  8. Test integration points

Scenario 2: Package Error During Build

Example: Error: Cannot find module 'drizzle-orm/pg-core'

Process:

  1. Identify package: Drizzle ORM
  2. ExternalScout: "Fetch Drizzle docs: PostgreSQL setup and imports"
  3. Check current import patterns
  4. Verify package.json has correct deps
  5. Propose fix from current docs
  6. Request approval → Apply fix

Scenario 3: First-Time Package Setup

Example: Setting up TanStack Query in Next.js

Process:

  1. Check install scripts: ls scripts/install/ | grep -r "tanstack\|react-query" scripts/
  2. ExternalScout: "Fetch TanStack Query docs: Installation, Next.js App Router setup w/ Server Components"
  3. Get: Install steps | Peer deps | Config files | Current patterns | Best practices
  4. If install script exists: Review → Run
  5. If no script: Follow docs for manual setup
  6. Implement → Test

Scenario 4: Version Upgrade

Example: Next.js 14 → 15

Process:

  1. ExternalScout: "Fetch Next.js 15 docs: Breaking changes and migration guide"
  2. Review breaking changes
  3. Identify affected code
  4. Plan migration steps
  5. Request approval → Implement → Test

Real-World Example: Auth Implementation

Task: "Add authentication w/ Better Auth to Next.js commerce site"

// 1. ContextScout: Project standards
task(
  subagent_type="ContextScout",
  description="Find auth and security standards",
  prompt="Find context files: Auth patterns | Security standards | Code quality | Project structure"
)
// Returns: security-patterns.md, code-quality.md

// 2. ExternalScout: Better Auth docs (MANDATORY)
task(
  subagent_type="ExternalScout",
  description="Fetch Better Auth + Next.js docs",
  prompt="Fetch Better Auth docs: Next.js App Router integration
  
  Focus on: Installation | App Router setup | Session mgmt | Route protection | Drizzle adapter
  Context: Adding auth to Next.js commerce w/ Drizzle ORM"
)
// Returns: Current installation | Integration patterns | Drizzle config | Working examples

// 3. Combine and implement
// - Better Auth patterns (from ExternalScout)
// - Security standards (from ContextScout)
// - Code quality standards (from ContextScout)
// = Secure, well-structured auth implementation ✅

Error Handling

Error Type Process
Package Installation ExternalScout: installation docs → Verify package name/version → Check peer deps → Propose fix w/ current steps
Import/Module ExternalScout: import patterns → Check current API exports → Verify import paths → Propose fix w/ current imports
API/Configuration ExternalScout: API docs → Check current signatures → Verify config format → Propose fix w/ current patterns
Build Errors Identify package → ExternalScout: relevant docs → Check known issues/breaking changes → Propose fix from current docs

Best Practices

Do ✅:

  • Check install scripts first on new builds
  • Always fetch current docs before implementing
  • Use ExternalScout proactively on new builds
  • Verify versions match (package.json vs docs)
  • Check breaking changes when upgrading
  • Read install scripts before running
  • Check prerequisites (databases, services, env vars)
  • Test integrations between packages
  • Cache useful docs in .opencode/context/development/frameworks/ if frequently used

Don't ❌:

  • Assume APIs based on training data
  • Skip version checks (breaking changes are common)
  • Ignore peer dependencies
  • Mix patterns from different versions
  • Run install scripts blindly
  • Skip environment setup

Troubleshooting FAQ

"When exactly should I use ExternalScout?"

ALWAYS use ExternalScout when working with external packages.

Triggers: User mentions library | import/require statements | package.json deps | build errors | first-time setup | version upgrades

Rule: If it's not in .opencode/context/, use ExternalScout.


"What if I already know the library?"

DON'T rely on training data - it's outdated.

Example: You think "I know Next.js, I'll use pages/" → Reality: Next.js 15 uses app/ → Result: Broken code ❌

Always fetch current docs, even if you "know" the library.


"How do I know if something is external?"

External libraries: npm/pip/gem/cargo packages | Third-party frameworks | ORMs/databases | Auth libraries | UI libraries

NOT external: Your project's code | Project utilities | Internal modules

Check: Is it in package.json dependencies? → External → Use ExternalScout


"Can I use both ContextScout and ExternalScout?"

YES! Use both for most features.

// 1. ContextScout: Project standards
task(subagent_type="ContextScout", ...)
// Returns: code-quality.md, security-patterns.md

// 2. ExternalScout: Library docs
task(subagent_type="ExternalScout", ...)
// Returns: Current Next.js docs, Drizzle docs

// 3. Combine: Implement using both

"What if ExternalScout doesn't have the library?"

ExternalScout has two sources:

  1. Context7 API (primary): 50+ popular libraries
  2. Official docs (fallback): Any library w/ public docs

If library not in Context7: Auto-fallback to official docs via webfetch

You don't need to worry - ExternalScout handles it.


"How do I write a good ExternalScout prompt?"

Template:

task(
  subagent_type="ExternalScout",
  description="Fetch [Library] docs for [specific topic]",
  prompt="Fetch current documentation for [Library]: [specific question]
  
  Focus on:
  - [What you need - be specific]
  - [Related features/APIs]
  - [Integration requirements]
  
  Context: [What you're building]"
)

Good prompts: ✅ Specific | ✅ Focused (3-5 things) | ✅ Contextual | ✅ Current

Bad prompts: ❌ Vague | ❌ Too broad | ❌ No context


"What if I get an error after using ExternalScout?"

Normal - errors happen. Process:

  1. Read error message carefully
  2. ExternalScout again w/ specific error:
    task(
      subagent_type="ExternalScout",
      description="Fetch docs for error resolution",
      prompt="Fetch [Library] docs: [error message]
    
      Error: [paste actual error]
      Focus on: Common causes | Solutions | Correct API usage"
    )
    
  3. Check install scripts (maybe setup incomplete)
  4. Verify versions (package.json vs docs)

"Do I need approval to use ExternalScout?"

NO - ExternalScout is read-only, no approval required.

Approval required: ❌ Write code | ❌ Run commands | ❌ Install packages

Approval NOT required: ✅ ContextScout | ✅ ExternalScout | ✅ Read files | ✅ Search code


"ContextScout vs ExternalScout?"

Aspect ContextScout ExternalScout
Searches Internal project files External documentation
Location .opencode/context/ Internet (Context7, official docs)
Returns Project standards, patterns Library APIs, installation
Use for "How we do things here" "How this library works"
Tools glob, read, grep webfetch, Context7 API
Speed Fast (local) Slower (network)
Currency Static (project docs) Live (current docs)

Use both together for best results.


"Quick Checklist: Am I doing it right?"

Before implementing w/ external libraries:

  • Used ContextScout for project standards?
  • Checked for install scripts first?
  • Used ExternalScout for EACH external library?
  • Asked for installation steps in ExternalScout prompt?
  • Asked for current API patterns and examples?
  • Specified what I'm building (context)?
  • Read returned docs before coding?
  • Following both project standards AND library docs?

All checked? → You're doing it right! ✅


Supported Libraries

See: .opencode/skill/context7/library-registry.md

Categories: Database/ORM (Drizzle, Prisma) | Auth (Better Auth, NextAuth, Clerk) | Frontend (Next.js, React, TanStack) | Infrastructure (Cloudflare, AWS, Vercel) | UI (Shadcn/ui, Radix, Tailwind) | State (Zustand, Jotai) | Validation (Zod, React Hook Form) | Testing (Vitest, Playwright)

Not listed? ExternalScout can still fetch from official docs via webfetch


References

  • ExternalScout Agent: .opencode/agent/subagents/core/externalscout.md
  • Library Registry: .opencode/skill/context7/library-registry.md
  • ContextScout Agent: .opencode/agent/subagents/core/contextscout.md
  • Code Standards: .opencode/context/core/standards/code-quality.md