The issue with the Azure API Key for the LLM model is a known bug that has persisted for quite some time. It is causing significant confusion and frustration for users, who often suspect their own environment before realizing that the problem lies with the API key.
If there is no official plan for a fix or support, it may be appropriate to consider shutting down this feature altogether.
FYI.
I tested a few cases.
Model: o3-mini: impossible
Model: o1: impossible
Model: gpt-4o: possible