Cursor (Claude 3.5 Sonnet) could be more accurate when asked to run tests for new changes

Cursor does great when adds new tests to my Rails feature specs. But, when I ask it to create a terminal command to run the tests for newly added changes (e.g. bin/rspec my_tests_spec.rb:111), it cannot figure out the line number to append after “:” in the command. That surprised me, I thought, it’s not a problem for a model to track the context of the hunks with changes it has just applied.

This can be surprising to know, but the underlying AIs within Cursor all have similar deficits around things like being able to correctly count how many letter 'R’s are in the word strawberry, as the AI doesn’t have any real understanding of the format or layout of the code is it given, or it’s our output.

For example, it’s unable to tell you the 5th letter in a word or count how many words were in it’s last message, as it just doesn’t have a method of actually running this kind of logic.

It does mean, for cases like yours, the AI probably won’t be any good at appending the correct line numbers unfortunately, but this will likely improve in the future as the underlying models do so.

1 Like