Why is adding docs so frustrating?

Feedback
1

The ability to index and reference docs is one of the primary reasons I use cursor.

Why is it so opaque and bad? I can never tell what page I should be using when adding them, and often no page will ever work to index more than just that exact url. Why is there no way to add more pages to the same custom doc when the scraping doesn’t work?

What magical combination of entrypoint and prefix will give me more than a single page from here:

At this point I feel like I don’t even know what entrypoint and prefix mean. Is the prefix not a common root that all indexed urls will share? Is the entrypoint the page from which it will find links with that prefix?

I hate trying to add docs because I know there’s a 50% chance it’s going to be frustrating and pointless.

Isn’t this just… web scraping? Can you please make it not fucking terrible?

2 Likes
2

Same. Genuinely no idea what can be so difficult about crawling a docs website, especially as simple as Docusaurus.

1 Like
3

Can’t even get anthropic docs to work!

1 Like
4

it is also oddly random for me. it can successfully crawl an md file in a github repo but fail on a different file at the same directory.