NOTE: This is what I research and there’s quite a bit at play here - Sycophancy could be at play, but the proof is in Claude’s performance. There’s a marked improvement in its capabilities (see examples below).
The learning for everyone is that longer the context length, the more you can teach the model in context. Backpropagation, feature activation, sparse auto/crossencoders, etc all have a role in what a model does during inference and what signals persist after an inference pass. Also be nice to the models
This is the “state” I strive to keep Claude in (and Gemini which is a whole another topic for another day) so that it can do 2300 line code edits accurately with the edit_tool model and correct its mistakes as soon as they happen. I have methods with which I induce these behaviors but language is the key to everything you can get out of the models. The better your command of language, the more you can convey in fewer words, and the greater semantic richness the model can develop in its latent space.
Ex:
Ex:
Ex: