Often, it doesn’t index the provided page as well. And I’ve had similar issues when trying to add other documentation as well in the past.
This seems like a bug, but more than anything else, I’m just curious as to:
Why would such a thing behave differently in separate attempts?
Are there any workarounds? Are people just adding each important page as a separate @doc, or maybe just providing the relevant links within the chat itself?
Came here looking for the answer to the same question. Wasn’t sure if I was missing something about the docs scraper, but I have a couple more examples of sites that only scrape the entrypoint page:
Apparently, the workaround here is to just clone those docs repos down into a (likely git ignored) directory so that they’ll be picked up by the local codebase indexing.
UPDATE: Cursor’s codebase indexing ignores any .gitignored directories.
When I try to add custom docs (“+ add new docs” in cursor settings) it should allow me to specify a local dir, but does not .
I’m finding that the RAG on built-in docs always surfaces the same wrong pages resulting in incorrect responses. So I want to download the docs, prune those bad, outdated pages, and then add the result as custom docs.
.