Curious to know if the Claude opus jobs submitted by cursor is faster than then jobs submitted by customized api key? Are there any optimization done in the backend as gpt models?
I also had Claude with anthropic api key in some kind of messy state and not working.
What I did:
Removed claude models by clicking the delete icon
Toggled Anthropic api key off and then on => models got added back there
(perhaps optional) For some reason I did not get these working before resetting my custom OpenAI base uri to default. Weirdly got OpenAI server error even if the selected model was from Anthropic.
May I ask how your experience with Claude in place of GPT 4 has been? When Cursor works, it has been great, but my biggest frustration with it so far is that it constantly disregards the codebase context (whether I provide a file, directory, class, or the entire codebase) and refuses to acknowledge that it can access those files. I’m wondering if Claude is more performant in that respect?
I tried Opus on openrouter with 40k context (short language reference) it is surprisingly good in understanding unknown new DSL/programming language (Verse from epic games). but in gpt4 api i am capped at 8k context for now (as a n00b tier user who still didn’t spent enough cash to be upgraded to next tier where access to gpt4 32k context will reveal itself… i am dreaming of the day when i’ll finally grind my level up to 128k access [ps it’s ridiculous ikr] but maybe one day i’ll be able to compare full “in-context” performance with Claude). Sonnet and Haiku are not as good, Haiku completely misses the point of the provided reference and doesn’t even understand the basic syntax outlined in it. the performance of GPT4 + RAG (uploaded files) is pretty good as well, but Opus nails it with precision (writing some new code based on completely unknown language, using only the in-context reference) while GPT4 does occasional mistakes. I believe the RAG search cannot be as good as full in-context data, when it’s about new concepts or syntax/facts. The cost with 40-45k tokens is 0.6$ per single request though (in Opus).
how is it when comes to completely new concepts in the RAG, is it able to find all the correct facts from multiple locations in the indexed documentation and codebase in order to answer a specific question or write code? or it’ll be provided with only a single chunk of information from all the indexed RAG data?
While experimenting with Cursor, I realized that these tools need further improvement, such as
"understand the request →
→ match request to all relevant ‘pages’ in the RAG db [I don’t care how it’ll be implemented but somehow this stage must locate ALL relevant pieces of information and also know where it ‘starts’ and where it ‘ends’ correctly, and extract it for next model] →
→ after all pages of info get collected across all indexed sources in the RAG db, pass the request and the joined info to the target model →
→ inspect the response and compare for ‘facts’ with existing data in RAG db [runs locally, sends as much api calls to gpt4/sonnet/whatever, for the verification process] →
→ if the content is not factual or wrong, re-phrase the request to the main model one more time, with augmented info by the findings of errors [same content as was in 1st request with RAG db info, but appended with ‘pay attention to possible mistakes you could make in XYZ and use ABC methods/functions to avoid the pitfall’ instruction], and repeat, until received content from the remote model will be high quality and will pass the local verification “factfulness/correctness”) →
→ PROFIT !
WDYT? isn’t it genius? although will cost quite a bit, in case of Opus/GPT4 combo for all of the above steps. BUT the quality will be outstanding. That thing can earn 150$/hr all by itself, it’ll cover the api costs
[ps it’s ridiculous ikr] but maybe one day i’ll be able to compare full “in-context” performance with Claude). Sonnet and Haiku are not as good, Haiku completely misses the point of the provided reference and doesn’t even understand the basic syntax outlined in it. the performance of GPT4 + RAG (uploaded files) is pretty good as well, but Opus nails it with precision (writing some new code based on completely unknown language, using only the in-context reference) while GPT4 does occasional mistakes. I believe the RAG search cannot be as good as full in-context data, when it’s about new concepts or syntax/facts. The cost with 40-45k tokens is 0.6$ per single request though (in Opus).
but somehow this stage must locate ALL relevant pieces of information and also know where it ‘starts’ and where it ‘ends’ correctly, and extract it for next model] → 
although will cost quite a bit, in case of Opus/GPT4 combo for all of the above steps. BUT the quality will be outstanding. That thing can earn 150$/hr all by itself, it’ll cover the api costs 
