The o1-mini model doesn’t seem to have streaming responses

Bug Report
1
  1. Open AI panel
  2. Toggle model to o1-mini
  3. Ask for help

The o1-mini model returns the full response at once, not gradually.

2

I suspect this may be due to using OpenRouter under the hood.

Big conclusion jump - however I use OpenRouter for my products and they announced that while streaming is supported, all the tokens come in at one time (for o1-preview and o1-mini specifically)

This is probably a wise decision as 20RPM rate limit from their standard OpenAI account would not serve all of us!

3

I have a question: It seems that rate limits are designed to alleviate server pressure by delaying request processing. For instance, with a 20 RPM limit, the server would process my request after a delay and return the first token, rather than sending all tokens at once.

If I’m misunderstanding how RPM works, please correct me.

4

OpenAI has not yet enabled streaming for those models through the API.

1 Like