After using Bugbot for awhile, I’ve noticed that it will often only report one issue in its comment, even if there are other issues present. Example scenario:
- PR has two unrelated problems, one in File A, one in File B
- Bugbot runs, and leaves a comment flagging the problem in File A, no mention of File B
- I push a commit fixing the problem in File A
- Bugbot runs again, and now it comments with the issue in File B
Why didn’t both of these get included in the first Bugbot run?
BugBot can totally comment multiple bugs per run! Would love to see the examples you’re talking about to see if we can improve it.
Happy to share! Here’s the most recent example.
Cursor first comment raising one bug in File A:
My commit fixing only the raised issue:
Then Bugbot runs again and raises a different issue in File B:
These were normal runs (not in Verbose mode) so I don’t know what the request IDs were, any way to obtain them after the fact?
No worries about the request ID, thanks for sharing! Will look into it. Overall there’s always going to be some variability here, plus we do a lot of work to try to reduce false positives. We could tweak some levers on our end that would make it more likely to show both bugs in one run, but that may also increase false positive rate which we are very apprehensive about. It’s all a tricky balancing act to get it perfect, and we’re constantly working on improving it.
Thanks for your feedback and we hope you’re enjoying BugBot!!
2 Likes
That makes sense! Preventing false positives is probably more important. We’re very impressed with Bugbot so far, it’s an amazing tool. Thank you for creating it!
2 Likes