Hey, thanks for the request. This is a known limitation. A few users have hit the same issue when trying to use multiple custom models with different endpoints at the same time.
A similar request was posted 8 days ago:
This feature is in progress. The team is working on fixing how “Override OpenAI Base URL” behaves with custom models. For now, the only workaround is to manually switch the Override Base URL in settings when you switch models.
I’ll add your use case to the existing thread so the team can see the demand.