Hey guys, first-time poster – and I have a head scratcher for you 
I’m indexing Shopify Polaris docs. Cursor is indexing pages that no longer exist, and not including some recent additions. It’s almost as if it’s using a cached version. Shopify Polaris v12 was released early October, but the indexed docs all refer to Polaris v11 (and Polaris v12 beta).
Some of the scraped/indexed docs no longer exist in the sitemap, or on the website (I looked through the HTML). Yet they appear as pages indexed by Cursor. And some new sections/pages added since the release of v12 isn’t among the indexed docs.
If I ask the docs “what’s the latest version of Polaris?” I get the answer:
The latest version of Polaris is version 12, which is currently in beta and scheduled for a stable release at the end of September.
Similarly, if I ask “What stack component exists?” I get:
- VerticalStack
- HorizontalStack
- LegacyStack
Both 1 & 2 above were replaced in Polaris 12, and are no longer in the sitemap or on the website. The list of indexed docs include links to components/layout-and-structure/horizontal-stack and …/vertical-stack, which now are 308 redirects to inline-stack and block-stack.
Where does the scraper/indexer find these? Can I do something to work around this?
Steps to reproduce:
- @add Components — Shopify Polaris /
- Name your docs, use URLs: same as above + without /components/
Try to find e.g. inline-stack or block-stack components, or anything about
Version: 0.13.4
VSCode Version: 1.83.1
Commit: fe924b35718993dc0821b4583f718e4cbe79b330
Date: 2023-10-29T00:38:06.904Z
Electron: 25.8.4
ElectronBuildId: undefined
Chromium: 114.0.5735.289
Node.js: 18.15.0
V8: 11.4.183.29-electron.0
OS: Darwin arm64 22.4.0
