tl;dr: I’ve noticed a uniform drop in competency across the models. From rules ignored more often, context not seeming to be considered, direct code references or instructions are ignored.
I’ve been using a form of local RAG before mcp arrived, with planning and agent mode as rules, found a GH repo called UltraContext and made a fork for my own preferences
Now this was a game changer.
However, since MCP’s agents and apparently models have improved, I’ve had to see why the agents are no longer able to complete basic requests any more.
I’ve tried running blank requests without the rule sets, I’ve tried migrating local memory file approach to the Memory MCP.
Where as I could ask the agents to create entire web apps, with SSR preloading, that was actually good and not just, making it complie, now I cant even get it to make copies of a few functions that already exist without it trying to refactor dependency files that I haven’t mentioned.
Annoyingly, the worst agents I’ve had are the premium 3.7 MAX & Gemini MAX agents.
On top of being an additional cost, they seem to do the least amount of planning or coding.
Am I missing something here?
I’ve run the same, duplicate a function, request on 3 models with and without my rules and its all basically wasted effort