Please add a setting to only automatically run the BugBot on every commit for PRs marked as ready for review and not run at all on Draft PRs.
3 Likes
I second this. I will pay for this once the free trial is up because we’ve already gotten good results. However, happening on drafts is a problem for us.
2 Likes
Will look into this, thank you for the feedback!
2 Likes
Would also love this functionality!
OP already said this but want to emphasize making this configurable instead of getting rid of this behavior altogether. The current behavior is nice for our workflow: have an automated pass by BugBot on drafts before human reviewers are added automatically by CODEOWNERS when it’s made “Ready to Review” by the author.
Guys any update on this?
I’ve started paying for BugBot but in 1-2 hrs. we’ve spent $30 because of it running on drafts. And this is a slow period. We’re not going to spend $10k / day for this but would gladly spend $1k / month once we have this filter.
cc @jkaplan
Hey! We turned it off on drafts like a week ago! So sorry, forgot to update this thread.
1 Like
@bericp1 You can always trigger a manual run by commenting “bugbot run”. Totally understand the desire for this to be configurable though, will think about it but can’t promise anything yet
Oh snap! Thank you!
…this seems like an expensive service for us! Blew through $50 today and that was only a partial day. 
Good stuff though. TY for this 
1 Like
Right now we’re seeing a median run cost of about $0.60 per run, but this is highly variable on the size of the PR and your codebase. We believe that the bugs BugBot finds are well worth the cost, but please let us know if you have any feedback!
It hallucinates about library-specific things sometimes (like how a toast function has been used), very similar to day-to-day Cursor usage. This leads to reporting bugs that aren’t really bugs.
Yeah, false positives are definitely a thing, we’re working hard on improving that but it’s an impossible problem to fully solve