Indexing custom doc only indexes home page

Bug Reports
1

When indexing a doc (faststream in my case) only the home page was indexed at first. then when clicking the refresh index button it starts actually crawling the doc. Eventually the result is “Failed indexing”
used url: FastStream - FastStream
![Screenshot 2025-02-16 at 14.31.00|690x166]

Screenshot 2025-02-16 at 14.32.22
Screenshot 2025-02-16 at 14.32.221155×402 38.8 KB

(upload://vAM439V4a5rLLBs4IIsubARzux8.png)

Version: 0.45.11

VSCode Version: 1.96.2

Commit: f5f18731406b73244e0558ee7716d77c8096d150

Date: 2025-02-07T09:43:58.555Z (1 wk ago)

Electron: 32.2.6

Chromium: 128.0.6613.186

Node.js: 20.18.1

V8: 12.8.374.38-electron.0

OS: Darwin arm64 24.3.0

2

Yeah this is a known issue with client-side websites and those with bot protection - FastStream’s docs may be built with client-side rendering which can cause indexing problems

For now you can try:

  1. Using a raw GitHub docs URL instead if available
  2. Waiting a few minutes between indexing attempts
  3. Using a different docs URL format (like their readthedocs URL if they have one)