Indexed 0 pages

Discussion
1

I see various topics/issues about indexing in the forum but I can’t say that any of those ‘solutions’ work or if there are ongoing issues with index. Anyways, I wanted to point out again issues with indexing.

I have a variety of repos I want to index so i can reference them with @Docs. I followed a few solutions regarding using the xml site and prefixing.

No matter how I play with this I always get Indexed 0 pages.

Here are two example repos that I can’t seem to index directly:

The first one (ajv-formats) is a strange one. I can’t seem to index that github link but this link works Ajv JSON schema validator. Even if I try to index just the README still doesn’t work.

Anyone have similar issues with github repos? What solutions are you using?
Thank you!

2 Likes
2

Seems like this has been an issue for a while. Here are a few other topics about this with recent activity (it won’t let me post more than two links unless I use a code block):

https://forum.cursor.com/t/doc-indexing-failed/31605
https://forum.cursor.com/t/the-docs-indexing-feature-is-not-working/52386
https://forum.cursor.com/t/index-docs-with-ajax-menu-indexed-0-pages/52574
https://forum.cursor.com/t/adding-documentations-that-dont-have-index-page-or-sitemaps/15171

Not sure what’s the best way to consolidate reports about this issue.

1 Like
3

Cursor_AbEYYqYert
Cursor_AbEYYqYert941×337 14 KB

I can’t index any documents. Is there any work being done to fix this issue? The main reason I subscribed was for the document indexing feature. I hope this disappointment will be resolved soon.

4

Hey, can you share the URLs to the documentation?

5

Hey.

https://docs.securityonion.net/en/2.4/
https://www.kok-emm.com/docs
https://docs.suricata.io/en/latest/

Version is 0.46.7.

6

I think it’s somehow related to the “Read the Docs” engine.

cc @danperks

1 Like
7

Is this a temporary issue, or has it always been like this?

8

Got another one here:

1 Like
9

Are you aware of this issue? @danperks

10

I am seeing this behavior as well, below is a list of URLs that result in “Indexed 0 pages” when adding them as custom docs:

https://docs.docker.com/build/buildkit/
https://docs.docker.com/build/exporters/
https://docs.docker.com/build/bake/
https://ui.wavv.com/

Most of the Docker docs will show up as having 0 pages indexed. How does Cursor’s indexing system work? If you open up the Docker Bake docs for example, it expands a tree of other pages on the left hand sidebar, each of these pages are nested under the docs.docker.com/build/bake/* URL path prefix.

There is no Table of Contents including links to each of these entries on the root /bake URL(the page I entered when adding a custom doc to Cursor), and as such, there aren’t any links on the page pointing to each one of these sub-pages(aside from the left sidebar which does have links to said pages under the /bake/* path prefix), so if Cursor is using the links in the page contents for indexing it would probably cause issues.

If we could get more info on how the indexer works, it might help documentation developers modify their docs in a way that is compatible with Cursor’s indexing, because as it is now, most sites don’t seem to work with the indexer.

1 Like
11

This feature seems like a total crapshoot. I got it working once but now everything I try is “indexed 0 pages”

12

Zero-page indexing has been an issue for me for a long time. It’s something to do with the way the documentation repository is formatted. @Docs is unable to traverse large sites where the index to the documentation is a panel on the left-hand side of the page with links. In some case I am able to index the panel one link at a time, but for the larger documentation set this is not feasible.

The @docs site traversal has to be broken into at least two steps for this to work better

13

Seems to be working for me now. Not sure if something changed. I’m on 0.47.5 FWIW.

14

What two steps should the process be divided into?

15

Step one is recognize some documentation sites have a navigation pane, usually on the left side of the screen. Step two, Set a loop that increments thru each subject item on the left pane, recursively read the entire subject, return and continue to the second subject, etc Alchemy.com has a documetation area that is laid out like that, and impossible to index