It would be nice to have some metrics/leaderboard about different models, derived from cursor usage data. For example, which models have the highest accept vs reject rate, especially if it’s possible to compare the results of the same request using different models.
Similar to lmarena, or Aider’s leaderboard or similar. Could be two categories – one based on standardized tests applied to all models, another based on real-life accept rates of cursor users using specific models.
Aside from helping decide which models are best, could also help cursor’s seo.
Thanks for the suggestion! We don’t currently have a way to track model performance metrics or acceptance rates across different models. While we do collect some basic usage data, we don’t have the infrastructure to compare model performance on identical requests
The idea of a leaderboard is interesting though - both for standardized tests and real-world usage. I’ll pass this feedback along to our team as we’re always looking for ways to help users make informed decisions about which models to use
For now, you can find info about our available models and their capabilities here: Cursor – Models