[Feature Request] Temperature optimization with internal benchmarks

Hi,

The decision has been made to set the temperature to 0.0.
However, some people on X indicate that setting a higher temperature can increase HumanEval / other benchmarks by a few percent.
Would it be possible for you to run each model with each temperature up and until 1.0 and choose the best temperature based on the results of the internal benchmarks?
If not, could we at least be able to overwrite the temperature of 0.0 with our own in the Beta/Alpha settings somewhere?

3 Likes