# Application Architecture

**URL:** <https://forum.cursor.com/t/application-architecture/328>\
**Category:** Discussions\
**Created:** [September 3, 2023, 2:19pm UTC](https://forum.cursor.com/t/application-architecture/328 "2023-09-03T14:19:09Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![my3sons](https://avatars.discourse-cdn.com/v4/letter/m/e274bd/32.png) [@my3sons](https://forum.cursor.com/u/my3sons)\
**Post date:** [September 3, 2023, 2:19pm UTC](https://forum.cursor.com/t/application-architecture/328/1 "2023-09-03T14:19:09Z")

</div>

For those of us that are just curious about these things, do you have any architecture diagrams/docs that you would be willing to share the sheds some light on how the sausage is being made behind the scenes? I get that if I am running “New AI Project” or “cmd-K” that those requests are presumably being directed to GPT-x via openAI calls, but what about interactions with Chat where we are asking questions about our codebase that exists only on our local machines, are you using LangChain or something like that to index all that code?

Thanks for a great product and keep up the good work!

---

<div class="post-metadata">

**Author:** ![ash](https://avatars.discourse-cdn.com/v4/letter/a/f19dbf/32.png) [@ash](https://forum.cursor.com/u/ash)\
**Post date:** [September 4, 2023, 8:34am UTC](https://forum.cursor.com/t/application-architecture/328/2 "2023-09-04T08:34:51Z")

</div>

> [@How does docs crawling work](https://forum.cursor.com/t/how-does-docs-crawling-work/264/3):
>
> We plan on adding a visualization of what it is the model actually sees when using docs in a future release. We’ll also have better docs management and auditing of text is shown for each page For now, we use an HTML to markdown parser, then do n-gram deduplication across the crawled pages (to take out useless info like navbars). Finally, we use some simple chunking heuristics to break down a page into multiple ~500 token chunks. The library name is node-html-markdown if you’d like to see what …

Doc Crawling Explained

---

<div class="post-metadata">

**Author:** ![system](https://sea3.discourse-cdn.com/cursor1/user_avatar/forum.cursor.com/system/32/99069_2.png) [@system](https://forum.cursor.com/u/system)\
**Post date:** [June 18, 2026, 10:10am UTC](https://forum.cursor.com/t/application-architecture/328/3 "2026-06-18T10:10:16Z")

</div>

Closed on June 18, 2026, 10:10am UTC