The updated Claude 3.5 Sonnet shows wide-ranging improvements on industry benchmarks, with particularly strong gains in agentic coding and tool use tasks. On coding, it improves performance on SWE-bench Verified from 33.4% to 49.0%, scoring higher than all publicly available models—including reasoning models like OpenAI o1-preview and specialized systems designed for agentic coding. It also improves performance on TAU-bench, an agentic tool use task, from 62.6% to 69.2% in the retail domain, and from 36.0% to 46.0% in the more challenging airline domain. The new Claude 3.5 Sonnet offers these advancements at the same price and speed as its predecessor.
6 Likes
Is cursor directing queries to the latest model automatically?
2 Likes
Is the new model being used by cursor yet?
This feature should’ve been long coming imo, but I really need to use BOTH the models served by Cursor Pro and custom API Key at the same time. You can’t switch quickly because you have to go to the settings each time. For now I want to use the best model for most complex cases but for easier tasks I’d like to use the previous Sonnet 3.5
4 Likes
the blog post does say it’s available today but i wonder if cursor will have to update the api endpoint?
They just have to change the model to use the latest version, I imagine they will do it soon.
does the new model need to be tuned for cursor?
It’s not following instructions properly. I’m asking it to create a markdown file for requirements and it ends up modifying other files instead.
Interesting, could you Cmd + P > Report AI Action, select the right request, and add a comment?
1 Like
For some reason it’s not showing up in the list since it’s been a couple hours and it’s only showing the most recent.
here I’m asking Composer to create a new SOP based on some prompt, but instead it goes and tries to edit route.ts
It happened at least four or five times consistently before I gave up
1 Like
Will we get Haiku 3.5 in Cursor soon? I’m burning through tokens so fast on Sonnet.
1 Like
2 Likes
That’s great news. Hopefully it is as great in coding as they say.
Knowledge cutoff still seems to be June 2023 for the claude-3-5-sonnet-20241022 (which was the previous model’s cutoff date). New claude-3-5-sonnet at https://claude.ai/ cutoff date is at April 2024. Are we sure this works?
1 Like
when will the long context claude also be updated to the new version? right now we only have claude-3-5-sonnet-200k which I assume is still pointing to the old version (?)
3 Likes
strongly hope the long-context new sonnet 3.5 will be added asap. Should be a no-brainer and free improvements?
New Claude Sonnet 3.5 has proven to be awesome in all my coding tasks so far. It’s just the “naming convention” or lack of it which is pretty stupid.
Waiting to see when Claude Sonnet 3.5 “new” long context is available.
1 Like