* fix(evals): repair six eval-framework behavioral regressions
Restore the deterministic eval baseline by fixing false-negatives and
identity/discovery bugs across the framework:
- error-handling: correct severity-boundary fixtures so mixed-severity
cases actually cross the >1000 warning threshold (deterministic
error/warning/info), plus explicit 1000-vs-1001 boundary tests.
- context-loading-evaluator: emit `no-context-loaded` and
`context-loaded-after-execution` as errors so missing/late context
fails (was a passing warning); update focused test accordingly.
- tool-usage-evaluator: enforce dedicated tools — bash `cat`/`ls`/`ls -la`
now fail as `bash-antipattern-*` errors (ls de-allowlisted); npm/git/
piped commands still pass.
- test-case-loader: canonicalize legacy agent ids via normalizeAgentId
(`openagent` -> `core/openagent`), preserving canonical/unknown ids.
- suite-validator: accept category-based agent ids, resolve test paths
under the canonical dir, add requested-vs-declared conflict detection;
new suite-validator.test.ts.
- validate-suites-cli: recursive suite discovery (nested <cat>/<agent>/
config), deterministic order, fail-closed on zero suites; new
validate-suites-cli.test.ts.
Deterministic Vitest allowlist green; validate:suites:all non-vacuous
and exits 0. No model/network/eval:sdk execution.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(evals): clear pre-existing TypeScript build errors (tsc 39 -> 0)
`npm run build` (tsc) failed on 39 pre-existing errors in files outside
the behavioral-repair scope, blocking the baseline gate. Root causes:
- glob: drop obsolete @types/glob ^8 so glob v13's own types (globSync)
resolve (fixes run-sdk-tests.ts).
- task-type-detector: introduce a local DetectedTaskType union for this
(dead/unused) detector; shared TaskType union left unchanged.
- behavior-evaluator: the expectedResponse check filtered a nonexistent
`message` event type and was silently dead — route through
base-evaluator.getAssistantMessages (assistant_message/text). Adds
three deterministic tests exercising the now-live check.
- integration tests: remove unused `expectedOutcome` fixture metadata
(not on TestCase, never asserted) and add the required `category`
field the excess-property error had masked.
tsc now exits 0. Vitest allowlist 112/112. Lint remains unconfigured in
evals/framework (no ESLint config has ever existed) and is deferred to a
separate follow-up; it is not part of this change.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(evals): sync bun.lock after @types/glob removal
The previous commit removed @types/glob via npm, updating package-lock.json
but leaving the primary bun.lock stale, which broke CI's
`bun install --frozen-lockfile`. Regenerate bun.lock so both lockfiles
agree (drops @types/glob and its transitive @types/minimatch only).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* ci(evals): add evals/framework/package-lock.json for build-check
The pr-checks `build-check` job caches on and runs `npm ci` in
evals/framework, but that workspace had no local lockfile, so the job
failed at Setup Node.js ("Some specified paths were not resolved") and
would fail `npm ci`. Add a standalone, in-sync package-lock.json
(lockfileVersion 3, no @types/glob) so the job's cache path resolves and
`npm ci` succeeds. Validated locally with `npm ci --dry-run`.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* feat(cli): migrate packages/cli to Bun-first with full test suite
- Replace fs-extra with Bun.file/Bun.write/node:fs/promises across all 9 source files
- Fix --help fast-path so all 8 subcommands are visible
- Replace tsup/tsx/vitest with bun build/bun run/bun:test
- Replace __dirname with import.meta.dir; use JSON import assertion for version
- Fix hardcoded OAC_VERSION in add.ts; add hashesMatch import to status.ts
- Replace computeFileHash with Bun.file().bytes() in sha256.ts
- Rename checkNodeVersion → checkBunVersion; parallelise doctor checks
- Fix TypeScript void-inference errors in update.ts, list.ts, status.ts
by replacing .catch() with let/try-catch where result is used downstream
- Add ManifestError named error class; remove unsafe type casts
- Add 43 bun:test unit tests (sha256, manifest, installer, version) — 0 failures
* fix(cli): resolve critical bugs and standards violations found in review
Critical bug fixes:
- ide-detect.ts: replace Bun.file(dir).exists() with stat().isDirectory()
for all directory checks — Bun.file().exists() always returns false for
directories, breaking Cursor/Windsurf/OpenCode/Claude detection entirely
- bundled.ts: add registry.json exclusion anchor to findPackageRoot() to
prevent monorepo root from matching before the CLI package root
Standards cleanup (§4.1, §5.1, §6.1, §15.3, §21.1):
- status.ts: parallelise findModifiedFiles + detectIdes with Promise.all
- apply.ts: fix duplicate warn/limit messages; remove else after return in
reportWarnings; remove redundant mkdir before Bun.write
- version.ts: remove unnecessary (pkgJson as {version?:string}) cast
- manifest.ts: remove redundant mkdir before Bun.write; remove as unknown cast
- installer.ts: remove redundant mkdir calls before Bun.write throughout
- add.ts: add comment explaining why node:fs/promises rm is used
Tests (43 → 142, +99 new tests across 4 new files):
- ide-detect.test.ts: 26 tests covering all 4 IDEs, both claude indicators,
detectIdes parallel, isIdePresent — directly validates the directory fix
- bundled.test.ts: 27 tests for classifyBundledFile, findPackageRoot,
listBundledFiles, getBundledFilePath, bundledFileExists
- config.test.ts: 25 tests for readConfig/writeConfig round-trips,
createDefaultConfig, mergeConfig, isYoloMode, isAutoBackup
- installer-update.test.ts: 14 tests covering all 5 updateFiles decision
branches (install/update/skip/yolo/dry-run) plus isProjectRoot
- sha256.test.ts: +4 tests for empty file, large file, binary content
* fix(cli): fix P0 bugs — global flags, .git detection, package root resolution
- Remove duplicate --dry-run/--yolo/--verbose from parent program; Commander.js
global option stealing caused all safety flags to be silently dropped in every
subcommand action callback
- Fix isProjectRoot() to use stat() for .git detection; Bun.file().exists()
returns false for directories, breaking oac init in standard git repos
- Add OAC_PACKAGE_ROOT env var override to getPackageRoot() for dev/monorepo mode;
registry.json heuristic excluded the repo root causing oac init/add/update to
throw when run from source
- Fix build script: remove --banner flag that caused double shebang in dist/index.js
- Rewrite bin/oac.js to invoke bun instead of node (dist is bun-only target)
- Refactor installer.ts let accumulators to const using Promise.all + reduce
- Add MVP planning docs (00-MVP-PLAN.md, master synthesis, project breakdown)
* fix(install.sh): handle both singular and plural component type formats
The get_registry_key() function was always adding 's' to the type,
causing 'contexts' to become 'contextss' which doesn't exist in registry.json.
Now handles:
- Singular forms: context → contexts, agent → agents, skill → skills
- Plural forms: contexts → contexts (unchanged), agents → agents
- Config stays singular
- Fallback for any type ending in 's'
Fixes#257
- Add TypeScript registry validator using Bun (< 1 second vs bash hanging)
- Install Bun in CI workflow
- Update workflow to use bun validator instead of bash script
- Add npm scripts for local validation
- Add comprehensive README for registry validation
This fixes the CI timeout issue where bash validator hangs in GitHub Actions.
The TypeScript validator is faster, more reliable, and easier to debug.
Related: Unblocks PR #128 which is currently failing due to bash script timeout
