Feature description
Selective allowance of models per individuals, though still respecting global enterprise-level settings, for specific override capabilities.
This would, in the end, and as an example, forbid the use of a given model (eg: Claude Opus) for the entire company though still allowing certain individuals (through override) access to it, for certain critical/complex tasks.
Feature requirements
- Respect global model blocklist or allowlist
- Add specific override to those into the user-level, either through the team management interface or on a new specific page for that on the dashboard
- Have support for similar configuration through API for DevOps and automation support
- [STRETCH] Support time limit for this allowance (eg: end date)
Justification
Some models, like Claude Opus, are notoriously more expensive than others, like GPT or Gemini. However, either due to regular practice, default configuration or sheer laziness, developers don’t tend to keep switching between models very often. Instead, once a model works they tend to stick to that model, permanently.
Although we can set a expense cap on both the enterprise level as well as at the user level, that’s not optimal. In this example, depending on what the user might be trying to do, they would end up spending their cap entirely, and then being throttled through the month, with little else to be done except an exceptional provisioning of a higher cap for their expense.
With this model blocking option, we can limit the options available unless there’s a proper justification, thus directly impacting the overall spending patterns and propagation of bad practices through developers and engineers.
In the end, this would work together with the cap, to ensure optimal model usage (from the allowed ones) while also providing safeguards so that the total spend is not beyond control, while also allowing for overrides depending on the work nature for each task assigned.