Spending tokens to repair UTF-16 that should have been UTF-8

[Sergio_Luiz_Nunes_Fa (Sergio Luiz Nunes Faria)]

Where does the bug appear (feature/product)?

Cursor IDE

Describe the Bug

On Windows, Cursor’s Agent (Write and StrReplace tools) often persists code files as UTF-16 LE when UTF-8 is expected. This is not fixed by standard VS Code/Cursor settings (files.encoding: utf8, files.autoGuessEncoding: false) or by .editorconfig with charset = utf-8.

The code looks correct in the chat, but on disk it breaks runtimes and tooling (Node.js, TypeScript, psql, etc.), causing rework, excessive token usage, and slower development.

Environment
OS: Windows 10/11 (e.g. Windows_NT x64 10.0.26200)
Cursor: recent version (Agent / Composer)
Stack: TypeScript, JavaScript, SQL, JSON (Node + PostgreSQL repos)
Workspace: local project on disk (not WSL-only)
Observed behavior
The Agent uses Write (new file or full overwrite) or StrReplace to edit code.
The file is saved as UTF-16 LE (alternating character + 0x00 bytes, e.g. 69 00 6D 00 70 00 for import).
When running the project:
Node.js: SyntaxError: Invalid or unexpected token at line 1, often showing only i (first byte of import in UTF-16 read as UTF-8).
TypeScript: invalid character errors.
psql: syntax error at the start of SQL files.
Chat output is fine; the issue is only on-disk file encoding.
What does not fix it (already tried)
“files.encoding”: “utf8” and “files.autoGuessEncoding”: false in .vscode/settings.json
.editorconfig with charset = utf-8
Manual save in the editor (after the Agent has already written UTF-16)
Project rules instructing “always UTF-8” — the Agent still writes UTF-16 for new files or large rewrites
Conclusion: the Agent’s write pipeline appears to bypass editor encoding settings and select UTF-16 LE on Windows.

Business impact (tokens and velocity)
Wasted tokens: each corrupted file triggers another Agent cycle to diagnose, explain, run conversion scripts, or rewrite the file — often 2–3× the tokens of the original task.
Extra verification step: after most Agent-created/edited files we must verify encoding (hex dump, node --check, tsc, psql, UTF-16 detection scripts).
False “done” state: the Agent reports success, but the app does not run until encoding is fixed.
Fragile workarounds: community uses hooks (afterFileEdit), avoid Write, copy UTF-8 templates, shell/Python writes, pre-commit fixes — all outside the product.
Community threads (same symptoms):

Windows: Cursor Write tool creates UTF-16 LE files instead of UTF-8

Where does the bug appear (feature/product)? Cursor IDE Describe the Bug On Windows 11, Cursor’s Write tool sometimes creates new files encoded as UTF-16 LE instead of UTF-8. This causes text to appear with spaces between every character (UTF-16 interpreted incorrectly). Restarted PC, still happens. Editing existing files doesn’t cause it, mainly happens when creating new files via Write tool. Workaround: creating files with PowerShell in UTF-8 works. Added .editorconfig + .vscode/settings.…

Spending tokens to repair UTF-16 that should have been UTF-8

Three weeks ago Cursor began producing corrupted UTF-16 encoded files when they should have been UTF-8. The thrashing from UTF-16 to UTF-8 and back is costing time, money and corrupting code, scripts and documentation. I stated a notebook to capture the issues and fixes. Cursor blames Windows, PowerShell, and the Cursor writer app while asserting that it cannot ensure all documents are produced as UTF-8, even when there are mandatory rules to do so. Here is the notebook for everyone reference…

Steps to Reproduce

On Windows, open a workspace with a Node/TypeScript project.
Ask the Agent to create a new .js or .ts file with the Write tool (or fully rewrite an existing file).
Inspect the file on disk:
Hex: XX 00 XX 00 pattern or FF FE BOM
Or: node --check path/file.js → SyntaxError at line 1
Convert the same content to UTF-8 (no BOM) → error goes away.
Real example: a controller .js file starts with 69 00 6d 00 70 00 6f 00 72 00 74 00 instead of 69 6d 70 6f 72 74 (UTF-8 import).

Expected Behavior

All Agent writes on Windows should use UTF-8 (ideally without BOM), consistent with files.encoding and the ecosystem (Node, npm, Git, PostgreSQL).
Write and StrReplace should follow the workspace encoding policy.
Users should not need extra Agent turns just to fix encoding corruption caused by the Agent itself.

Operating System

Windows 10/11

Version Information

Version: 3.6.31 (user setup)
VS Code Extension API: 1.105.1
Commit: 81fcf2931d7687b4ff3f3017858d0c6dee7e2a60
Date: 2026-05-31T17:46:29.630Z
Layout: editor
Build Type: Stable
Release Track: Default
Electron: 39.8.1
Chromium: 142.0.7444.265
Node.js: 22.22.1
V8: 14.2.231.22-electron.0
xterm.js: 6.1.0-beta.220
OS: Windows_NT x64 10.0.26200

For AI issues: which model did you use?

Agent (Composer) with model set to “Auto” on Windows.

For AI issues: add Request ID with privacy disabled

I am not disabling Privacy Mode and I do not have a Request ID to share. I am not sure what “Request ID with privacy disabled” means in practice, and I prefer to keep privacy settings as they are.

The issue is not occasional: it happens dozens of times per day in real development work. Almost every Agent session that creates or heavily edits files (.ts, .tsx, .js, .sql) on Windows risks writing UTF-16 LE instead of UTF-8. I then have to detect the corruption (Node/TypeScript/psql errors), run conversion scripts, or ask the Agent again — which wastes a large amount of tokens and time.

I can reproduce on demand: ask Agent to Write a new .js/.ts file and inspect hex or run node --check. Cursor version: [paste from Help → About Cursor → Copy].

Additional Information

This is a daily, high-volume problem for me — not a one-off. I use Agent all day with Auto. Each corrupted file often costs an extra Agent turn (diagnosis + fix), so the token and time impact is severe across a full workday.

Does this stop you from using Cursor

No - Cursor works, but with this issue

[mohitjain (Mohit)]

Confirming this is still an open issue we’re tracking - it isn’t fixed in 3.6.31, and I don’t have a timeline to share.

Until the fix lands, the most reliable mitigations (depending on what’s getting corrupted):

• Edits to existing files: enable the Windows UTF-8 locale - Control Panel > Region > Administrative > Change System Locale > check “Beta: Use Unicode UTF-8 for worldwide language support”, then restart.

• New files: the locale setting doesn’t cover the Write tool path, so the most reliable workaround is a Cursor Rule that tells the agent to create new files via terminal commands instead of the Write tool. @153653 posted the exact rule text above and reports it working without issue.

I know the repeated rework is frustrating - it’s a high-priority item for us.

[153512]

This bug is extremely disruptive! Please fix it now as it makes it very hard to work. It is wasting time and money which is not fair. The previous workarounds do not work anymore.

[153512]

I am officially completely fed up with this. It is nearly impossible to use Cursor and we should not be charged for this non-functional service. I now spend 75% of my time dealing with this nonsense.

@153512

Until the fix lands, the most reliable mitigations (depending on what’s getting corrupted):

Edits to existing files: enable the Windows UTF-8 locale - Control Panel > Region > Administrative > Change System Locale > check “Beta: Use Unicode UTF-8 for worldwide language support”, then restart.

New files: the locale setting doesn’t cover the Write tool path, so the most reliable workaround is a Cursor Rule that tells the agent to create new files via terminal commands instead of the Write tool. posted the exact rule text above and reports it working without issue.

THESE NO LONGER WORK!!

[barnej (Jakub Iwiński)]

Unfortunately I have the same issues. Why is this not fixed? I dont event want to know how many $$ in tokens i lost in rewriting files to utf-8…

[user477]

This is a nightmare to work with. Tasks that should take a minute take an hour. This needs to be fixed promptly! Why is it still not fixed? This is not fair to paying users.

[user477]

These workarounds no longer work! You are referring to workarounds that I originally posted. THEY DO NOT WORK! Cursor does not work essentially. I can’t create new files and I can barely edit existing files. This has gone on long enough, it’s ridiculous to work like this and waste so much time and money!

• Until the fix lands, the most reliable mitigations (depending on what’s getting corrupted):

• Edits to existing files: enable the Windows UTF-8 locale - Control Panel > Region > Administrative > Change System Locale > check “Beta: Use Unicode UTF-8 for worldwide language support”, then restart.

• New files: the locale setting doesn’t cover the Write tool path, so the most reliable workaround is a Cursor Rule that tells the agent to create new files via terminal commands instead of the Write tool. @153653 posted the exact rule text above and reports it working without issue.

[mohitjain (Mohit)]

Thanks for the continued reports, and sorry this is still disrupting your work. Quick status: this is still a known, high-priority issue we’re tracking, and I don’t have a fix timeline to share yet.

A few of you mentioned the earlier workarounds have stopped working. That can actually be two different things, and knowing which one helps us a lot:

• New files still get corrupted - unfortunately expected. The Windows UTF-8 locale setting never covered the Write tool’s file-creation path, so the Cursor Rule that creates new files through the terminal instead of the Write tool is still the most reliable mitigation there.
• Edits to existing files now get corrupted too - if existing-file edits were staying intact with the UTF-8 locale enabled and have started breaking, that’s a new variant we want to capture.

If you’re hitting that second case, a quick reply with your exact Cursor version (Help > About) plus the first line of Format-Hex -Path .\yourfile.ext -Count 16 run on one corrupted file would tell us whether this is the same bug or something new, and it gives the team concrete bytes to work from.

[user477]

@mohitjain

• the Cursor Rule that creates new files through the terminal instead of the Write tool is still the most reliable mitigation there.

That is what we are saying!! THIS does not work any more. Nothing works. I’m sorry guys but this is inexcusable. A disabling bug since the beginning of April and nothing. I am deeply dissatisfied. Meanwhile you are rolling out new features left and right and the basic core functionality of the software has been broken for 75 days!

Edits to existing files now get corrupted too - if existing-file edits were staying intact with the UTF-8 locale enabled and have started breaking, that’s a new variant we want to capture.

At first the windows settings was effective. Not at least 20% of the time it is not and edits fail too.

[user477]

Writing the updated files via shell since direct edits aren’t matching the on-disk encoding.

and that fails, for existing files, it edits them and changes them to utf-16, corrupting files across my codebase and wasting time and money. I am absolutely sick of this!! At least 75% of my work time today lost to waiting for failed edits and file creations.

[user477]

typical feedback aft er Composer edits a pre-existing file that was working fine

Cause: prizeGallery.js was saved as UTF-16 LE (each ASCII character followed by a null byte: 28 00 66 00 75 00… = (function…). The browser loads scripts as UTF-8, so after the opening ( it hits \0 at column 2 — hence:

[mohitjain (Mohit)]

Brett, thanks for posting the raw bytes. 28 00 66 00 75 00 is UTF-16 LE for (fu..., which is exactly what we needed to see, and I’ve made sure it’s captured for the team. You’re right that this is now hitting edits to existing files, not just new-file creation, and I’m sorry it’s still costing you this much time.

Two things that should help right now:

Recover the corrupted files. UTF-16 LE is fully reversible, so the content isn’t lost. For a single file, click the encoding indicator in the bottom-right status bar, choose Reopen with Encoding → UTF-16 LE (it’ll render correctly), then click it again and choose Save with Encoding → UTF-8.

Catch corruption automatically while the fix is pending. Since the per-tool workarounds (locale setting, terminal-create rule) are slipping for you, the most reliable approach left is a post-edit step that re-saves anything written as UTF-16 back to UTF-8 after every edit. The notebook at the top of this thread sets this up as a Cursor hook (afterFileEdit / postToolUse); because it runs on the result, it catches the problem whether Write, StrReplace, or a shell command produced it. If you batch-fix existing files, commit first so you have a clean restore point.

[Nick_Williams]

Has a fix been introduced for this entire class of corruption yet?
It has been reported many many times and is causing massive workflow issues for us (and lots of others apparently) - why hasnt this been prioritised over every other feature? It seems that an agent that can’t write files with appropriate encoding is a major issue for a code editor?

[user477]

It is ridiculous that this has not been fixed. You’re charging people for a product that cannot perform the most basic and fundamental tasks. $60 billion and you don’t even have a working product. Going on three months now!

[user477]

Even worse Cursor has started randomly “updating” and “fixing” vast numbers of my files that are completely outside the scope of the UTF issue. I will create a new file, it fails due to this now inexcusable bug, then it automatically attempts to fix it, and in the course of doing so just randomly decides to apply the UTF “fix” and re-save to huge swaths of my code base. The other day I created one new file, only to find more than 3,000 files re-saved on disk. There is no excuse for this. Ridiculous.

[user4251]

FWIW, this has been an issue for over a month and a half for me; quite disruptive and surprised there is no fix yet.

[user1590 (Eric Rickard)]

The UTF-8 issue has been known for over 2 months now and the only fix that the entire Cursor community has is to continuously expend time and tokens writing mis-encoded files that are then detected and converted to UTF-8. $M’s of customer funds are being spent to fix what Cursor should have fixed by now.

But that is not the worst of it. This single bug is the root cause of runtime errors that are cost ing more than 10x the lost customer productivity costs in system failures and lost profits.

This must be escalated to a Tier 1 fix immediately.

[user477]

Can another staffer respond to this and tell your paying customers when exactly this is going to be fixed, and that needs to be very soon.

[Jason_Luo1 (Jason Luo)]

Encoding bugs are especially painful because they turn into token waste twice.

First the agent creates or touches the wrong file encoding, then another run has to spend context and tool calls diagnosing something that should have been a write time invariant. For Windows heavy repos, I’d want encoding validation to be part of the edit path, not just a cleanup step after things break.

[mohitjain (Mohit)]

Thanks for the continued reports, and I’m sorry this is still eating into your work. Straight answer: it’s still an open, high-priority issue on our end, and I don’t have a fix date to share yet.

@user477 - the behavior in your latest posts (one new file kicking off a re-save of thousands of files across the repo) is actually separate from the core encoding bug, and it’s the part I’d most want to stop for you:

• Get a restore point first. If those mass re-saves happened during an agent run, you can roll them back with checkpoints. If the files were committed beforehand, git restore . also works - that discards uncommitted changes, so check git status first. Don’t let the agent keep “fixing” on top of an already-corrupted state.
• Check for a recursive post-edit hook. If you’re running the postToolUse encoding hook shared earlier in this thread, it scans the repo recursively and can re-save everything on a single edit. Scoping it to only the changed file usually stops the repo-wide re-saves.

That mass re-save is a different problem from the write-path encoding bug this thread tracks, so could you open a separate thread for it with a short screen recording (or the command the agent ran right before it happened)? It’ll get tracked properly there instead of getting lost in here.

On the encoding bug itself: I know the earlier workarounds have gotten less reliable for some of you, and that’s captured. UTF-16 LE is at least fully recoverable - click the encoding indicator in the status bar, Reopen with Encoding → UTF-16 LE, then Save with Encoding → UTF-8. We’ll post here as soon as the fix ships.