I have been trying to add custom documentation for Tamagui UI (Tamagui UI — Tamagui — style library, design system, and UI kit for React (Native and web)), but it fails to index any pages beyond the first page provided, even though all the other pages are clearly linked on the first page, and they all follow the same prefix (https://tamagui.dev/ui).
Often, it doesn’t index the provided page as well. And I’ve had similar issues when trying to add other documentation as well in the past.
This seems like a bug, but more than anything else, I’m just curious as to:
- Why would such a thing behave differently in separate attempts?
- Are there any workarounds? Are people just adding each important page as a separate @doc, or maybe just providing the relevant links within the chat itself?
3 Likes
Came here looking for the answer to the same question. Wasn’t sure if I was missing something about the docs scraper, but I have a couple more examples of sites that only scrape the entrypoint page:
Apparently, the workaround here is to just clone those docs repos down into a (likely git ignored) directory so that they’ll be picked up by the local codebase indexing.
UPDATE: Cursor’s codebase indexing ignores any .gitignored directories.
1 Like
When I try to add custom docs (“+ add new docs” in cursor settings) it should allow me to specify a local dir, but does not 
.
I’m finding that the RAG on built-in docs always surfaces the same wrong pages resulting in incorrect responses. So I want to download the docs, prune those bad, outdated pages, and then add the result as custom docs.
1 Like
same here can’t index any docs
when i need to index some docs, i search for the github page, identify the docs and use uighub.com to scrape. try using this link for tamagui:
https://uithub.com/tamagui/tamagui/tree/master/code/tamagui.dev/data/docs
