I often want to have multi-turn conversations in the chat mode. I’ll often make sure that my first turn contains a lot of specification and has added context from all the relevant files.
By default, follow-up turns in a given chat conversation should include context from the same files as earlier in the conversation. Otherwise, it can get very annoying very quickly. I’m not sure if you already do this under the hood.
This also gets to another feature request I made - I wish we had more clarity on what is actually getting included in the context window for a given inference call. That will make it a lot easier for power-users to prompt compose.
Hey, after checking the SQLite database (which they use to store chats), I’ll make an educated guess: when you start a chat and include a bunch of files or context in that first message, Cursor actually remembers all of that for the rest of the conversation.
1 Like
Oh interesting! How do you know they use a sqlite database? Is that in application’s installation directory or something?
Chats (on Windows) are stored here: “C:\Users\Bruger\AppData\Roaming\Cursor\User\globalStorage\state.vscdb”
On Mac: “~/Library/Application Support/Cursor/User/globalStorage/state.vscdb”
It’s an SQLite database that has:
key
value
The key is the chat ID, while the data is the full structured data for the chat.
I use DB Browser to view the database. It’s pretty useful to know this actually - it allowed me to remove messages from a chat that weren’t relevant (and taking up the context window). For example, wrote an SQL query to delete certain messages while persisting others.
Hey, when you first start a prompt and add context with the @ symbol, that context is kept in the following message, however, there are two caveats:
- We do not send the whole file as context to the composer,but rather snippets and chunks of the most relevant sections of the files. This means if you change what section of the file is needed as context, you should @ it again to ensure the AI has the right code to reference.
- As your conversation grows, the AIs ability to see the old context diminishes, so its good to regularly start new composer sessions to ensure it stays on topic and fresh in its responses.
When you say it’s ability to see the old conversation diminishes, are you referring to limitations in the overall context window or is it a consequence of the way you construct context windows? But this is really helpful! It sounds like you’re essentially saying that it’s almost always better for compose sessions to be single-turn as much as possible?
Related to another feature request I made, would it please be possible for users to directly see what goes into a given inference call? This level of insight would make debugging prompts sooo much easier! Almost like a debug mode that shows you the raw string that gets sent to Claude, and then perhaps overlays of the filenames, etc that those strings came from?
We do some magic behind the scenes to try to allow your conversations to be longer than the context window, but this does come with some limitations, hence my suggestion to start new composer sessions when you can. As you use Cursor more and more, you can begin to tell when the older context is being ignored and when it is time to start a new session.
Not sure we’ll add this level of visibility into the client as we don’t want to cause too much confusion for most of our users, but I agree that visibility is not a bad thing here!
Is the file @ just sending the most relevant parts of the file as slices and blocks? This is not a good idea, and it’s even quite bad. No wonder the results are always very random. It feels like he understands it one moment and doesn’t the next. It’s confusing. I think it’s fine for some design documents @ like this because the meaning won’t change. If it’s @ code, this strategy is really terrible. There should be some means to debug, like using certain methods to extract the blocks of the @ code files I need to mention. I don’t believe he calculates the most relevant parts as slices and blocks, especially for code. If this is true, then I guess we can’t just @ code casually to prompt it. I always thought it was giving the entire file to LLM, and I wrongly blamed Claude; it seems like an investigation is needed.
There is also an implicit issue here, because if the @ is a snippet or block of code, it is obvious that I am likely to modify it, which could completely misalign my context. It’s a very poor experience.
Yeah FWIW I also assumed that when I explciitly @ a file, it would send all of it to Claude.
I think it really would be quite helpful to get far more visibility into how the context window is being constructed.
The issue of confusing users is interesting. I think it depends on what segment of the market Cursor wants to target. I can see how for “beginners” or “no code” users it would be quite overwhelming. But I suspect that existing experienced engineers would actually appreciate the overall visibility and control over what’s going on under the hood.
I’ve personally found compose quite frustrating at times because I have no way to debug bad inference results. It’s impossible to tell whether I’m holding it wrong, whether it’s the RAG or whether it’s Claude. For everything else in my development stack, I’m used to just digging into how the implementation works.
Early on, it’s possible and likely that we do send your whole file to Claude, as the context window is empty and has space to do so, but as it fills up, we have to be more opinionated on what we send to Claude.
We’ve tested this extensively, and have found what works best so far, but we have some improvements shipping soon to enable a higher context window, so keep an eye out for that!