I’m having some problems indexing Haystack documentation. I’ve seen a couple threads about documentation indexing problems, but other than updating to the latest version of Cursor (which doesn’t help) I don’t see a fix.
What I’m trying to add: Introduction to Haystack 2.x. Indexing actually seems to progress: I can see different pages being indexed, but at some point it fails (with no log output). There are indeed pages indexed, but there’s still a failed message. And I can’t quite check if everything has been indexed, as the pages are not in order and there are a lot of them.
When I check the ‘Cursor Indexing & Retrieval’ logs in the terminal output, there is nothing related to indexing documentation. Adding a new documentation source, restarting indexing, etc don’t lead to any new log entries, even when setting the verbosity to trace.
Hey, adding the URL https://docs.haystack.deepset.ai/docs/intro worked for me, and is already indexed by our system. Can you confirm this was the URL you were trying, and the prefix was correctly identified?
Ah I see. Well if it shows a failure that means that at least some of the doc pages have failed to load. I hope the opcoming improvements make this a bit more robust. Thanks for your response!
