Adding Docs, Indexing keeping fails after a bit

Kindly request the Cursor team to index these popular packages.
I’m not sure if I’m spending my premium tokens by repeatedly trying to index these docs, which progresses a bit then fails.
These will be of use to almost all python programmers, so I shouldn’t have to do spend my quota for it :slight_smile:

https://docs.pola.rs/api/python/stable/reference/index.html

(Yes both of the below are required)
docs dagster io
docs-preview dagster io

Please acknowledge and add these.

When I initially added, they all showed last indexed dates in either 2023 or mid 2024. These packages improve fast and having Cursor attempt deprecated techniques or not knowing about better implementations wastes time for everyone who uses these packages.

Hey, document indexing doesn’t use your quota. I just added the link you provided, and after some time, I was able to complete the indexing. Currently, 1341 pages have been scanned, and it finished successfully. Regarding the incorrect version in the model output, I think it’s because it’s not specified anywhere in the documentation. However, if I click on any of the pages in the documentation, it redirects to the correct version.

1 Like

Thanks @deanrie. I see the same 1341 pages for polars but it still says Indexing failed.
For DuckDB, it says this after 1903 pages.

What do you think about doing a simple poll here, or on X, or over mail w/ your users - asking for docs that they want indexed. Then, it will save a lot of time for individuals trying to index docs manually.
Or, maybe an (open source?) component from Cursor org, that allows users to submit artifacts from their run at indexing (showing what they indexed, etc.).

EDIT: I dont’ mean product version from docs. I mean last indexed date that I see in the Docs section:

image

Hey, yes, despite the “Indexing Failed” message, the documentation should work. This is an error we plan to fix. Also, you can submit a PR to us for the documentation you need.

Thank you, done. Pls see Update docs.jsonl by v1gnesh · Pull Request #74 · getcursor/crawler · GitHub

Also, any thoughts on Ability to add indexed document collection (PDX) to Docs please?