Hi @whe and @rfeinman, could you try using the pyrefly or ty extensions?
pyrefly
ty
These language servers are implemented in Rust, so I would expect their performance to be much better. However, they do not yet support the same configuration options.