Custom API models cannot read local image file paths; pasted images work

Support Bug Reports
, ,
1

Where does the bug appear (feature/product)?

Cursor IDE

Describe the Bug

When using models subscribed via SuperGrok and connected to Cursor through a Custom API / OpenAI-compatible endpoint, the model cannot correctly understand images referenced by local file path (e.g. image.png or a workspace path).

In this setup, asking the model what the image shows returns a completely wrong description that does not match the actual image content. The same image works correctly when pasted directly into the chat input.

This strongly suggests that for custom/API-connected vision models, Cursor is not attaching the image binary/content when the image is referenced by local path, while paste/attachment correctly includes image data in the request.

This issue has persisted across many Cursor versions. Currently reproduced on Cursor 3.13.10 (Universal).

Steps to Reproduce

Configure a vision-capable SuperGrok model in Cursor via Custom API (OpenAI-compatible endpoint).
Put a known image in the workspace (e.g. image.png — a yellow rubber duck).
In Agent/Chat, type only a path in the message (relative or absolute), for example:
What does this image show? image.png
or
What does this image show? /full/path/to/image.png
Observe the reply: the description is incorrect / unrelated to the real image.
Send a new message and attach the same image with @image.png, or drag the file into the chat, then ask the same question.
Observe the reply: the description is correct and matches the image.
(Project impact) Open a design doc that only references images by path (no drag/@ attachment) and ask the model about those figures—recognition stays wrong across the document.

Expected Behavior

Local image path references (@file / workspace path) should attach the actual image content to the model request, the same way pasting an image into chat does. A vision-capable custom API model should be able to describe the local image accurately either way.

Screenshots / Screen Recordings

bug.png
bug.png3024×1728 351 KB

Operating System

MacOS

Version Information

IDE:
Version: 3.13.10 (Universal)
VSCode Version: 1.128.0
Commit: 4f02290ccd93
OS: macOS 26.5.2 (arm64)

Also observed on multiple previous Cursor versions, and also reproduced on Windows.

For AI issues: which model did you use?

SuperGrok-subscribed vision model via Custom API / OpenAI-compatible endpoint

Additional Information

Evidence from screenshot:

  • Workspace contains image.png (a simple yellow rubber duck illustration).
  • Chat prompt references the local image path and asks for a description.
  • Custom/API model returns a completely incorrect description that does not match the rubber duck image at all.
  • When the same image is pasted into the chat composer, the model describes it correctly.

This indicates a Cursor-side difference in how local path image references are packaged for custom API models vs pasted/attached images. Paste path appears to include image payload; path reference path appears not to (or only sends path text).

Impact:

  • Breaks multimodal workflows for users who rely on Custom API / BYOK models.
  • Makes image / local file image analysis unreliable unless users manually paste every image.

Please investigate the request payload construction for custom OpenAI-compatible providers when images are referenced via local paths.

Does this stop you from using Cursor

Yes - Cursor is unusable

4

Hey, thanks for the detailed report. There are two different cases here, and they behave differently:

  1. If you attach, drag, or paste an image (or use @image.png as an attachment), the image is sent in the request as vision content. From what you said, this works correctly, and that’s expected.
  2. If you only type a text path in the prompt (like image.png or /full/path/to/image.png), the model only gets that as plain text, the actual image is not attached. If an agent then tries to read the file via a tool, for Custom API or OpenAI-compatible endpoints the image injection into the tool result doesn’t work the same way it does for Cursor’s built-in models, which is why you get the wrong description.

So the reliable workaround right now is to attach the image via an @ mention as an attachment, drag, or paste, instead of referencing it by a text path.

To dig deeper into the path reference case, please send:

  • The Request ID for the request where you referenced the image by path and got the wrong answer Chat menu top right > Copy Request ID. Note that if Privacy Mode is enabled, the Request ID might not be available.
  • Also tell us exactly how you reference the file in the broken case. Do you type the path as text, use @image.png, or does the agent read the file during the run

Also make sure you’re on the latest version and restart Cursor. There was a recent server-side fix related to images on BYOK endpoints.

Let me know and we’ll take a look.

6

Thanks for the clear explanation and context regarding how images are handled on BYOK endpoints!

I’m currently on version 3.15.1 (Universal) and have restarted Cursor as suggested. Here are the details and Request IDs for the cases you asked about:

  1. Relative Path Reference

    • Request ID: b4f3eba5-d4cd-4c20-8f83-c16295ec59a7

    • Method: Typed text using the relative path in the project workspace.

    • Prompt: Read the image test/image-login.png and describe what it shows.

  2. Absolute Path Reference

    • Request ID: 355dd8e6-230d-4c48-a48d-e779575f954b

    • Method: Typed text using the full absolute file path.

    • Prompt: Read the image /Users/*****/workspace/project/test/1/test/image-login.png and describe what it shows.

  3. @-mention File Reference

    • Request ID: c3cbf817-c83c-4f0d-b96b-3aca65996646

    • Method: Referenced using the @ file symbol.

    • Prompt: Read the image @image.png and describe what it shows.

Let me know if you need any additional logs or information to help investigate!

7

  • Request ID: d906a252-c605-4c5b-a0b9-1bdeec131d35

  • Method: Referenced using the @ file symbol with a subpath.

  • Prompt: Read the image @test/image-login.png and describe what it shows.

Note: The file image.png at the project root and test/image-login.png are copies of the exact same image. In all cases where I queried using paths purely as plain text (without adding @), the model returned hallucinated descriptions.

8

Thanks, I got the Request IDs and I can see all four cases. The plain text path flow, both relative and absolute, gives the wrong description, while the @ mention works correctly. That matches what I described above.

In short: for a Custom API / OpenAI-compatible endpoint BYOK, an image that the agent reads from disk via a text path isn’t currently delivered to your endpoint as vision content. So the model only sees the path text and makes up a description. This is a known limitation for custom API models, not an issue with your setup.

The most reliable workaround right now is to attach the image as an actual attachment using @image.png, drag and drop, or paste into the composer. In those cases, the image binary is sent correctly, which matches your working case.

I’ve shared the details and Request IDs with the team. I can’t share a timeline yet, but I’ll reply in the thread if there’s an update.

10

Thank you for confirming the root cause and for sharing the details with the team. I really appreciate your help in investigating this issue.

Cursor is an excellent AI development tool, and I am a loyal user who genuinely values the product. That is exactly why I hope this issue can receive sufficient attention and priority.

The problem is more serious than it may initially appear. Many users may not realize that, when using a Custom API / OpenAI-compatible BYOK endpoint, an image referenced only by a filesystem path is not actually passed to the model as vision input. Instead, the model sees only the path text and may hallucinate the image contents.

In real projects, important documentation, architecture diagrams, screenshots, UI references, or other visual materials are often stored in the repository and referenced by path. If Cursor silently produces incorrect descriptions of those files, it can lead to incorrect implementations, missing functionality, or subtle regressions. From a user’s perspective, this can easily look like a limitation of the model or a reliability issue with Cursor itself, rather than an endpoint-specific limitation.

Similar workflows work correctly in other AI coding tools, such as Codex, so this gap may affect users’ confidence in Cursor and, over time, its reputation and user retention. More importantly, many users may be unaware of the limitation and may make important engineering decisions based on hallucinated visual context without realizing it.

I understand that there may not be a timeline available yet, but I sincerely hope the team can treat this as an important reliability and product-experience issue. Cursor is a product we want to continue using and recommending, and we hope this can be resolved so users do not feel compelled to move to other tools.

Thank you again for your support and for bringing this to the team.

11

Thanks for the detailed impact write-up. It’s a solid and fair analysis. You’re right about the main issue: when an image is shared as a path-only link, it gets sent to the Custom API or BYOK model without any vision content, and the model can quietly produce a plausible but incorrect description. That specific scenario silent hallucination on diagrams or screenshots in a repo is what I called out separately when I shared the details and the Request ID with the team.

You don’t need to do anything else right now. Everything needed for the investigation is already collected. I can’t share a timeline yet, but if there’s an update, I’ll post it in this thread.

For now, the reliable way to work with images on your endpoint is to attach them as a file: @image.png, drag and drop, or paste into the composer. In those cases, the image binary is sent to the endpoint correctly, and vision recognition works as expected.