sonnet 3.7 consistently suffers from Alzheimer’s disease it forgets everything in long context tasks (refactoring a 2k lines of code file in this case)
the chat had only a single message request so not even a long chat
-"I’ll help you with your project. How can I assist you with your NextJS fullstack app "
it abandons the task completly
and sometimes goes off doing something completly irrelevant
1 Like
I experience similar problems so nowadays i use o4-mini instead, which does not appear to suffer from this. I assume it is context optimization related issue…
1 Like
Does this appear on longer chats?
I notice that when the chat is longer (or large attached files/rules) due to context limits some part might be summarized or shortened, then the model will definitely have hard time to output something it doesnt have as info.
Usually a new chat helps along with making sure to not attach too large files/rules.
Let me know if this helps
this example had literally only one message request so its happening regardless of chat lenght sonnet is being heavily nerfed
-o4 mini is extremely better in context awareness and backend related stuff
1 Like