What’s going on behind the scenes? The composer feels like it has the memory of a goldfish and the foresight of a groundhog.
It’s hard to ignore what seems to be the signs: reduced context length and deliberate throttling. Is this an intentional limitation, or is the system collapsing under its own weight?
Users deserve honesty. If there’s a technical issue, own it and explain. If it’s a deliberate downgrade, at least have the decency to justify the decision.
Do not come with “Did you try to start a new composer?”.
User: Hi.
Composer: Hello.
Start new Composer.
User: Refactor this code according to this detail, the relevant files are in this folder.
Composer: print(“Hello, World!”)
User: >:(
1 Like
Hey, I can assure you there’s been no downgrade to how we process your requests behind the scenes, nor do we think there is a technical issue here, as none of the core components that make this work have changed significantly.
Do you have any examples, ideally accompanied with a screenshot, of where you see the composer misbehaving?
@danperks Thank you for your response. I’d like to provide specific details to illustrate the issue, but most of the time, it’s difficult to share examples due to the sensitive nature of the information involved. That said, I plan to take the time to create a test case for demonstration purposes in the future.
Based on my experience with Composer, this seems to be a context length issue. The model appears to retain a very general and abstract understanding of the context but often forgets specific details or nuances. On average, instructions or information shared three to four messages earlier are completely forgotten—particularly when dealing with topics that have multiple aspects.
Once I have the time, I will create and share a test case with the community. For now, I’d encourage you to consider that, if “none of the core components that make this work have changed significantly” (and I assume context length is included in this), it’s reasonable to infer that the implementation of features like agent mode—which increases the amount of information (tokens) shared with the model during each interaction—might be contributing to a noticeable decline in output quality.
While we are always looking to improve things on our end, the context drop of you are seeing is likely just a side-effect of LLMs in general, as the more context they receive, the less focus and attention they seem to place on it in their responses.
Unfortunately, we don’t have a good solution yet for “infinite context”, where we are able to point the LLM to the right chunk of your chat history, even if it was days or weeks ago, as context for the very latest prompt, which is why we often do throw out the easy fix of starting a new chat. I do understand that is a clean slate, and you then end up having to rebuild your context and chat history.
1 Like
agree it needs to fix all stuff with a new updates
