Agent audit regression: Composer 2.5 Fast PASSes on behavioral-only runs while edge suites find blockers

Hey, thanks for the detailed write-up. We rarely see reports this clean and well structured.

Key point: what you’re describing isn’t a product bug, it’s model behavior. Cursor doesn’t have built-in verdict gating. PASS is just text the model generates, not a mechanical check. Same deal with rules and AGENTS.md, they act as steering, not fail-closed constraints. The model can ignore them, especially in long sessions. Your hypothesis 1 completion bias, stopping once pytest tests/behavioral is green, and hypothesis 2 git ls-tree correctly excluding untracked tests/edge/ and the agent honestly skipping them, both match how it works. The second one is git semantics, not an agent bug. git ls-tree only shows what’s in the index, untracked files won’t appear there.

On Fast: Composer 2.5 and Composer 2.5 Fast are the same model with a serving toggle you switch via Edit in the model picker, not two different weights. The difference is latency and cost, not quality, so it probably doesn’t explain any difference in audit discipline. Looks like you already found the toggle.

What actually gives you mechanical enforcement today is Hooks. What you asked for in constructive suggestions, blocking PASS until the required commands ran, should be done with hooks, not prose rules:

  • beforeShellExecution to gate shell commands or require specific suites to run, fail-closed via exit code 2
  • stop to intercept agent completion and block a verdict if the edge suites weren’t run

There’s also a generic preToolUse that can fail on all tool calls, but for shell commands it’s better to use beforeShellExecution.

Docs: Hooks | Cursor Docs

There’s also a community guide called Model Is the Auditor with ready-made self-review hooks for Windows and Linux: [Guide] Thin Self-Review Hooks for Cursor | The Model Is the Auditor (Windows + Linux)

A similar behavior class is discussed here. It also breaks down the fail-closed pattern with exit code 2, which matches your case:

What worked for you, explicit per-round instructions like do not PASS until round N is green, is the recommended approach. Use hard gates in the loop, not in the rules text. Hooks formalize this so you don’t have to repeat it every session.

I’ll pass your feedback about the audit preset and warnings for unrun tests/edge/** to the Agent Quality team. Let me know if anything doesn’t work when you try hooks.