OpenWriterControle/.opencode/context/development/principles/api-design.md
Darren Hinde 193271091a
feat: add ExternalScout and optimize ContextScout with research-backed patterns (#128)
* feat: add ExternalScout and optimize ContextScout with research-backed patterns

## New Features

### ExternalScout Subagent (v2.0.0)
- Fetches live, version-specific documentation for external libraries via Context7 API
- Supports 18+ libraries: Drizzle, Better Auth, Next.js, TanStack, Cloudflare Workers, AWS Lambda, Vercel, Shadcn/ui, Radix UI, Tailwind CSS, Zustand, Jotai, Zod, React Hook Form, Vitest, Playwright
- Lazy-loads query patterns for specific libraries only
- Filters and formats results by relevance (Critical → High → Medium)
- Fallback to official docs via webfetch
- 42% token reduction via research-backed optimization
- Added to registry with skill:context7 dependency

### Context7 Skill
- Registered in system for external documentation fetching
- Library registry with 18+ supported libraries
- Query optimization patterns per library
- Navigation guide and README

## Optimizations

### ContextScout v5.1.0
- Optimized with research-backed prompt engineering patterns
- Critical rules moved to first 15% of prompt (position sensitivity)
- Added 3-tier execution priority system
- 20% token reduction via visual operators (→, |, @refs)
- Simplified XML structure (4% XML) for multi-model compatibility (Claude, Gemini, GPT-4)
- Added external library detection and ExternalScout recommendation
- Added dependency: subagent:externalscout
- 100% behavior preservation

### ExternalScout v2.0.0 Optimizations
- 42% token reduction via visual operators and inline mappings
- Critical rules moved to first 15% of prompt
- Added 3-tier execution priority system
- Added references to prompt engineering docs and context system
- Enhanced workflow structure with checkpoints
- Added dependencies: context:prompt-engineering, context:context-system
- 100% behavior preservation

## Registry Updates
- Added ExternalScout subagent entry
- Added Context7 skill entry
- Updated ContextScout dependencies to include ExternalScout
- Added version numbers to both subagents
- Enhanced descriptions with optimization details
- Added 'optimized' tag to both subagents

## Files Added
- .opencode/agent/subagents/core/externalscout.md
- .opencode/skill/context7/README.md
- .opencode/skill/context7/library-registry.md
- .opencode/skill/context7/navigation.md
- .opencode/agent/subagents/core/contextscout-v4-backup.md (backup)
- .opencode/agent/subagents/core/externalscout-v1-backup.md (backup)

## Files Modified
- .opencode/agent/subagents/core/contextscout.md (v4.0.0 → v5.1.0)
- registry.json (added ExternalScout, Context7, updated ContextScout)
- CHANGELOG.md (documented all changes)
- VERSION (0.5.5 → 0.6.0)
- package.json (0.5.5 → 0.6.0)

## Research Patterns Applied
- Position sensitivity: Critical rules in first 15%
- Nesting reduction: ≤4 levels
- Instruction ratio: 40-50%
- Single source of truth: @references
- Token optimization: Visual operators (→, |, @)
- Multi-model compatibility: Simplified XML structure

## Breaking Changes
None - 100% backward compatible

## Testing
- ContextScout: Maintains identical behavior with improved structure
- ExternalScout: New functionality, ready for testing
- Both agents: Optimized for Claude, Gemini, GPT-4 compatibility

Closes #[issue-number-if-applicable]

* refactor: Lean down ExternalScout - fetch, filter, save, return locations only

- Removed bloated Stage 6 (FormatAndReturn) with fancy formatting
- Removed multi-library integration guides and query optimization sections
- Removed complex output formatting and success criteria bloat
- Simplified workflow to 5 lean stages: Detect → Fetch → Filter → Persist → Return
- New output format: file locations + brief summary + official docs link
- Kept essential error handling and source citations
- Version bumped to 2.0.0
- Focus: Fast, accurate documentation retrieval without extra bloat

* feat: Add ExternalScout and optimize context system

- Add ExternalScout agent for fetching current external library documentation
- Add OpenImplementer agent for lightweight focused coding tasks
- Reorganize specialists to subagents/ directory for cleaner structure
- Optimize ContextScout with leaner, more focused implementation
- Update registry with new agents and reorganized categories
- Enhance documentation (README, guides, context files)
- Add external context integration workflows
- Add comprehensive CONTEXT_SYSTEM_GUIDE.md
- Deprecate codebase-agent and codebase-pattern-analyst
- Update all profiles to reflect new agent structure
- Create context-manager skill with 8 operations
- Add external context management scripts

BREAKING CHANGES: None (all changes backward compatible)

Fixes: Outdated training data issues for external libraries

* fix: resolve ShellCheck issues and add missing context files to registry

- Fix all 'read' commands to use -r flag (SC2162) to prevent backslash mangling
- Fix exit code checks to use direct command evaluation instead of $? (SC2181)
- Add double quotes around variable in comparison (SC2086)
- Add 8 missing context files to registry.json:
  - external-libraries.md (new workflow for external library handling)
  - visual-development.md
  - development/README.md
  - learning/README.md
  - product/README.md
  - data/README.md
  - mastra-ai/README.md
  - context-bundle-template.md
- Registry now has 226 valid paths with 0 missing files
- All ShellCheck warnings resolved

* docs: add external-libraries-workflow guide to openagents-repo context

- Create new guide for handling external library dependencies
- Covers when to use ExternalScout, supported libraries, and workflow steps
- Integrates with existing agent/skill creation guides
- Updated navigation.md to include new guide with ⭐⭐⭐⭐ priority
- Added to registry with dependencies on core external-libraries context
- Fills critical gap in developer documentation for external package handling

* fix: resolve ShellCheck SC2145 errors in context router

- Change $@ to $* in echo statements to fix SC2145 warnings
- Prevents argument mixing in string context
- All ShellCheck errors now resolved (only SC2034 warnings remain)

* fix: resolve all ShellCheck warnings in manage-external-context.sh

- Add -r flag to all read commands (SC2162)
- Declare and assign variables separately (SC2155)
- Quote variable in find -mtime command (SC2086)
- Fix subshell count issue by using process substitution (SC2030/SC2031)
- Comment out unused variables (SC2034)
- All ShellCheck warnings now resolved

* fix: remove unused variables in context router to pass ShellCheck

- Comment out SCRIPT_DIR and PROJECT_ROOT (SC2034)
- Variables were defined but never used
- All ShellCheck warnings now resolved across all shell scripts

* fix: replace bash registry validation with Python script

- Create fast Python validation script (validate-registry.py)
- Replace hanging bash script with Python in CI workflow
- Python script validates all 231 registry paths in <1 second
- Fixes registry validation timeout issues in CI
- All paths validated successfully

* chore: trigger CI re-run for workflow updates

* Revert "fix: replace bash registry validation with Python script"

This reverts commit 03843935d075f1350afb721b5235e9406005810f.

* fix: remove verbose flag and add timeout to registry validation

- Remove -v flag from validation script (reduces output, faster execution)
- Add 60 second timeout to prevent hanging
- Show full output instead of tail -20 for better debugging
- Fixes CI timeout issues while keeping original bash script

* feat: add TypeScript registry validator with full skill support

- Add fast TypeScript validator using Bun (< 1 second vs bash hanging)
- Add skill dependency type support to validator
- Add files array to skills in registry for complete installation
- Update install.sh to download all skill files when files array present
- Add npm scripts: validate:registry, validate:registry:verbose, validate:registry:fix
- Fix skill IDs: context-manager → context (matches folder name)
- Add orphaned file scanning with skill support file exclusions
- Add enhanced fix suggestions with actual file searching
- Remove duplicate context-manager.md file
- Add comprehensive README for registry validation scripts

Skills now install completely with all supporting files:
- SKILL.md (main definition)
- router.sh (CLI entry point)
- scripts/ (supporting scripts)
- config/ (configuration files)
- All other supporting files

Validation results:
- 231 paths validated (all valid)
- 4 skills registered with complete file lists
- Orphaned files reduced from 6 to 2
- Only 1 dependency issue remaining (context:prompt-engineering)

* fix: use TypeScript validator in CI instead of hanging bash script

- Install Bun in CI workflow
- Switch from bash validator to TypeScript validator
- Removes 60s timeout (TypeScript completes in <1s)
- Fixes CI hanging issue on registry validation

* chore: retrigger CI checks

Empty commit to retrigger CI after workflow fix in PR #129

* fix: use official Bun setup action in CI

- Replace manual Bun installation with oven-sh/setup-bun@v2
- Fixes PATH issue where bun wasn't available in subsequent steps
- Ensures Bun is properly installed before running TypeScript validator

* chore: retrigger CI after Bun setup fix in PR #131

Empty commit to retrigger CI once PR #131 is merged to main

* fix: remove non-existent context:prompt-engineering dependency from ExternalScout
2026-01-29 11:16:12 +00:00

8.8 KiB

API Design Patterns

Category: development
Purpose: REST API design principles, GraphQL patterns, and API versioning strategies
Used by: opencoder, openimplementer


Overview

This guide covers best practices for designing robust, scalable, and maintainable APIs, including REST, GraphQL, and versioning strategies.

REST API Design

1. Resource-Based URLs

Use nouns, not verbs:

# Bad
GET  /getUsers
POST /createUser
POST /updateUser/123

# Good
GET    /users
POST   /users
PUT    /users/123
PATCH  /users/123
DELETE /users/123

2. HTTP Methods

Use appropriate HTTP methods:

  • GET - Retrieve resources (idempotent, safe)
  • POST - Create new resources
  • PUT - Replace entire resource (idempotent)
  • PATCH - Partial update (idempotent)
  • DELETE - Remove resource (idempotent)

3. Status Codes

Use standard HTTP status codes:

2xx Success
  200 OK - Successful GET, PUT, PATCH
  201 Created - Successful POST
  204 No Content - Successful DELETE

4xx Client Errors
  400 Bad Request - Invalid input
  401 Unauthorized - Missing/invalid auth
  403 Forbidden - Authenticated but not authorized
  404 Not Found - Resource doesn't exist
  409 Conflict - Resource conflict (e.g., duplicate)
  422 Unprocessable Entity - Validation errors

5xx Server Errors
  500 Internal Server Error - Unexpected error
  503 Service Unavailable - Temporary unavailability

4. Consistent Response Format

Standardize response structure:

// Success response
{
  "data": {
    "id": "123",
    "name": "John Doe",
    "email": "john@example.com"
  },
  "meta": {
    "timestamp": "2024-01-01T00:00:00Z"
  }
}

// Error response
{
  "error": {
    "code": "VALIDATION_ERROR",
    "message": "Invalid input data",
    "details": [
      {
        "field": "email",
        "message": "Invalid email format"
      }
    ]
  },
  "meta": {
    "timestamp": "2024-01-01T00:00:00Z",
    "requestId": "abc-123"
  }
}

// Collection response
{
  "data": [...],
  "meta": {
    "total": 100,
    "page": 1,
    "pageSize": 20,
    "totalPages": 5
  },
  "links": {
    "self": "/users?page=1",
    "next": "/users?page=2",
    "prev": null,
    "first": "/users?page=1",
    "last": "/users?page=5"
  }
}

5. Filtering, Sorting, Pagination

Support common query operations:

# Filtering
GET /users?status=active&role=admin

# Sorting
GET /users?sort=createdAt:desc,name:asc

# Pagination
GET /users?page=2&pageSize=20

# Field selection
GET /users?fields=id,name,email

# Search
GET /users?q=john

6. Nested Resources

Handle relationships appropriately:

# Good - Shallow nesting
GET /users/123/posts
GET /posts?userId=123

# Avoid - Deep nesting
GET /users/123/posts/456/comments/789
# Better
GET /comments/789

GraphQL Patterns

1. Schema Design

Design clear, intuitive schemas:

type User {
  id: ID!
  name: String!
  email: String!
  posts: [Post!]!
  createdAt: DateTime!
}

type Post {
  id: ID!
  title: String!
  content: String!
  author: User!
  comments: [Comment!]!
  publishedAt: DateTime
}

type Query {
  user(id: ID!): User
  users(filter: UserFilter, page: Int, pageSize: Int): UserConnection!
  post(id: ID!): Post
}

type Mutation {
  createUser(input: CreateUserInput!): User!
  updateUser(id: ID!, input: UpdateUserInput!): User!
  deleteUser(id: ID!): Boolean!
}

input CreateUserInput {
  name: String!
  email: String!
}

input UserFilter {
  status: UserStatus
  role: UserRole
  search: String
}

2. Resolver Patterns

Implement efficient resolvers:

const resolvers = {
  Query: {
    user: async (_, { id }, { dataSources }) => {
      return dataSources.userAPI.getUser(id);
    },
    users: async (_, { filter, page, pageSize }, { dataSources }) => {
      return dataSources.userAPI.getUsers({ filter, page, pageSize });
    }
  },
  
  User: {
    posts: async (user, _, { dataSources }) => {
      // Use DataLoader to batch requests
      return dataSources.postAPI.getPostsByUserId(user.id);
    }
  },
  
  Mutation: {
    createUser: async (_, { input }, { dataSources, user }) => {
      // Check authorization
      if (!user) throw new AuthenticationError('Not authenticated');
      
      // Validate input
      const validatedInput = validateUserInput(input);
      
      // Create user
      return dataSources.userAPI.createUser(validatedInput);
    }
  }
};

3. DataLoader for N+1 Prevention

Batch and cache database queries:

import DataLoader from 'dataloader';

const userLoader = new DataLoader(async (userIds) => {
  const users = await db.users.findMany({
    where: { id: { in: userIds } }
  });
  
  // Return in same order as input
  return userIds.map(id => users.find(u => u.id === id));
});

// Usage in resolver
const user = await userLoader.load(userId);

Frontend API Client Patterns (TanStack Query)

Use TanStack Query for optimal client-side API consumption:

REST Integration

// Optimal REST client with TanStack Query v5
import { useQuery, useMutation, useQueryClient } from '@tanstack/react-query';

const apiClient = {
  getUsers: (filters) => 
    fetch(`/api/v1/users?${new URLSearchParams(filters)}`).then(r => r.json())
};

function UsersList() {
  const { data, isPending, error } = useQuery({
    queryKey: ['users', filters],
    queryFn: () => apiClient.getUsers(filters),
    staleTime: 5 * 60 * 1000, // 5 minutes
  });

  return (
    <div>
      {isPending && <div>Loading...</div>}
      {error && <div>Error: {error.message}</div>}
      {data?.data.map(user => <UserCard key={user.id} user={user} />)}
    </div>
  );
}


## API Versioning

### 1. URL Versioning

**Version in the URL path**:

GET /v1/users GET /v2/users


**Pros**: Clear, easy to route  
**Cons**: URL changes, harder to maintain multiple versions

### 2. Header Versioning

**Version in Accept header**:

GET /users Accept: application/vnd.myapi.v2+json


**Pros**: Clean URLs, flexible  
**Cons**: Less visible, harder to test

### 3. Deprecation Strategy

**Communicate deprecation clearly**:
```javascript
// Response headers
Deprecation: true
Sunset: Sat, 31 Dec 2024 23:59:59 GMT
Link: <https://api.example.com/v2/users>; rel="successor-version"

// Response body
{
  "data": {...},
  "meta": {
    "deprecated": true,
    "deprecationDate": "2024-12-31",
    "migrationGuide": "https://docs.example.com/migration/v1-to-v2"
  }
}

Authentication & Authorization

1. JWT Tokens

Use JWT for stateless auth:

// Token structure
{
  "sub": "user-123",
  "email": "user@example.com",
  "role": "admin",
  "iat": 1516239022,
  "exp": 1516242622
}

// Middleware
function authenticateToken(req, res, next) {
  const token = req.headers.authorization?.split(' ')[1];
  
  if (!token) {
    return res.status(401).json({ error: 'No token provided' });
  }
  
  try {
    const decoded = jwt.verify(token, process.env.JWT_SECRET);
    req.user = decoded;
    next();
  } catch (error) {
    return res.status(401).json({ error: 'Invalid token' });
  }
}

2. Role-Based Access Control

Implement RBAC:

function authorize(...roles) {
  return (req, res, next) => {
    if (!req.user) {
      return res.status(401).json({ error: 'Not authenticated' });
    }
    
    if (!roles.includes(req.user.role)) {
      return res.status(403).json({ error: 'Insufficient permissions' });
    }
    
    next();
  };
}

// Usage
app.delete('/users/:id', 
  authenticateToken, 
  authorize('admin'), 
  deleteUser
);

Best Practices

  1. Use HTTPS everywhere - Encrypt all API traffic
  2. Implement rate limiting - Prevent abuse and ensure fair usage
  3. Validate all inputs - Never trust client data
  4. Use proper error handling - Return meaningful error messages
  5. Document your API - Use OpenAPI/Swagger or GraphQL introspection
  6. Version your API - Plan for breaking changes
  7. Implement CORS properly - Configure allowed origins carefully
  8. Log requests and errors - Enable debugging and monitoring
  9. Use caching - Implement ETags, Cache-Control headers
  10. Test thoroughly - Unit, integration, and contract tests

Anti-Patterns

  • ❌ Exposing internal IDs - Use UUIDs or opaque identifiers
  • ❌ Returning too much data - Support field selection
  • ❌ Ignoring idempotency - PUT/PATCH/DELETE should be idempotent
  • ❌ Inconsistent naming - Use camelCase or snake_case consistently
  • ❌ Missing pagination - Always paginate collections
  • ❌ No rate limiting - Protect against abuse
  • ❌ Verbose error messages - Don't leak implementation details
  • ❌ Synchronous long operations - Use async jobs for long tasks

References

  • REST API Design Rulebook by Mark Masse
  • GraphQL Best Practices (graphql.org)
  • API Design Patterns by JJ Geewax
  • OpenAPI Specification (swagger.io)