OpenAI / ChatGPT BYOK models abort after brief reply in Cursor

Thanks a lot for taking the time, I hope this reply helps:

I dug into the traffic on my side and can confirm the ApplyPatch custom tool isn’t surviving the round trip. My endpoint sits in front of an upstream that speaks the newer Responses-style API, so requests get translated on the way there and the reply gets translated back into chat/completions for Cursor. The translation back is where it breaks:

  • The request going upstream is fine — the type:"custom" ApplyPatch tool passes through and the model does call it. Upstream returns a valid custom_tool_call.
  • But on the return translation, only standard function_call outputs get mapped into tool_calls. Anything that isn’t a function_call — i.e. the custom_tool_call — is silently dropped. Cursor gets back finish_reason:"stop" with content:null and tool_calls:null, so the edit just stalls.
  • Everything else works because all the other tools (Shell, read, search, etc.) are ordinary type:"function" tools, which translate correctly. ApplyPatch is the only type:"custom" one, so it’s the only thing that fails. That isolation made it easy to spot once I had logs.

Answering your questions as best I can without getting too into the weeds on my specific setup:

  • What’s serving it: a self-hosted forwarder in front of an upstream OpenAI-compatible provider (not OpenRouter).
  • Endpoint: Cursor hits /chat/completions.
  • Is it translating between the two APIs: yes — that’s the crux. It translates /chat/completions → the upstream’s Responses-style API and back. The forward direction handles custom tools fine; the reverse direction has no handler for custom_tool_call and drops it.
  • Endpoint logs: yes to both halves of your question — the ApplyPatch custom_tool_call does appear in the response the forwarder receives from upstream, but it does not appear in what gets sent back to Cursor. It’s lost precisely in the responses→chat/completions translation.

One thing worth flagging for anyone hitting this: it seems to have started when ApplyPatch moved to the grammar-based type:"custom" format. Setups that previously sent the edit tool as a plain function tool wouldn’t have seen it, which probably explains why it’s only surfacing now for a subset of custom-endpoint users. The fix on the forwarder side is exactly what you said, relay type:"custom" tools through unchanged and map the returned custom_tool_call back verbatim instead of dropping unrecognized tool-call types.