Docs in Cursor: two different behaviors

[younio]

Hello,

I would like to understand how Docs indexing work:

From my experience with Cursor, when I insert a URL:

• sometimes, it would index the “root page” and its children in the documentation (which is great, and what I would actually expect) => example from the Rust doc of a crate
• sometimes, it would index just that specific page only, why? ==> example of the tSQLt documentation (it only crawled/indexed one page) => I had to copy/paste tens of URLs into Cursor individually.

I would like more details on how this mechanism works and why it acts differently.

[danperks]

@younio

tSQLt

Hey, I had a good at indexing these docs and it looks like Cursor doesn’t auto-detect the prefix correctly, so you need to make sure you fix it for indexing to work correctly!

For example, you can input the page:
https://tsqlt.org/user-guide/test-creation-and-execution/newtestclass/
but you will need to fix the prefix to be:
https://tsqlt.org/user-guide/
so Cursor knows it can look at any URLs started with that prefix.