Using .txt files as documentation/context + increasing token limit

[Senhua]

The doc crawling feature is already amazing and one of the benefits of using cursor. That being said, for situations where it is buggy, I was wondering if in the future we could either upload documents or allow us in a .txt file to paste all documentation and then have cursor use that documentation information to help provide relevant context to generate more helpful code for things that are after the cutoff date.

Currently, I am not sure how cursor handles things when documentation takes up a majority of the tokens or even exceeds 8k.

[amanrs]

For now, a solution would be opening a new file in the editor (Cmd+n), pasting contents into that file, and finally using @file to tag it.

[HappyQuokka]

how can we see how many tokens what uses? I fed about 500 pages of docs into it and not sure if I did correctly

[Jakob]

@HappyQuokka I don’t think you can see how many tokens were used (except in your OpenAI dashboard if you use your own API key). But if the file exceeds Cursor’s token limit it will only use the most relevant chunks from the file and display them in the “Long-file Details” dropdown in Chat.

[HappyQuokka]

It seems there is some indexing across files, that gets referenced and finds the correct ones to put into context - as I see the chat responds with “used page X, used page Y” if I tag ‘all docs’, so the system tries to find the correct pages from the huge list which I reference? I wonder if someone from the Cursor can chime in and shed light on this advanced ‘behind the scenes’ black magic . But suggestions it makes are kind of 50/50, sometimes it hallucinates over the exact same pages it just said “I referenced them” so it tries to chew them during response to query but not always correctly

very promising if it was working in a way that info from these docs is 100% utilized in generating the answer so it won’t be able to contradict the docs or ‘not notice’ the knowledge provided in them.

[HappyQuokka]

by the Cursor token limit you mean current context of the model which they use?

[HappyQuokka]

this approach is literally the same as pasting into chatgpt chat session, correct? that’s a ‘worst case scenario’ kind of workflow, for people who are looking in a new software to find ‘something better than chatgpt’. But it’s definitely a working method for small things and small snippets, just that we’re looking into Cursor AI as a savior from this exact workflow which we already use with chatgpt .