Edit Here is a similar question but is specifically about request sync block in async. I'm just curious more about why the concurrency means they all take substantially longer - at different places in the stack.
We are running Django with uvicorn asgi in kubernetes. Following best practice guides we are doing this with only 1 worker, and allowing the cluster to scale our pods up/down.
We chose asgi as we wanted to be async ready, however currently our endpoints are all sync.
Internally we are using our own Auth (micro service) which is a request to an internal pod using Pythons request library. This works via a JWT being passed up which we validate against our public keys then fetch User details/permissions. After this, it's all just ORM operations: a couple of .get() and some .create()
When I hit our endpoint with 1 user this flies through at like 20-50ms.
However as soon as we bump this up 2-5 Users, the whole thing comes to a grinding halt. And the requests start taking up to 3-5s. Using profiling tools we can see there's odd gaps of nothing between the internal Auth request finishing and then going on to do the next function. And similar in other areas.
To me this seems to be simply a concurrency issue. Our 1 pod has 1 uvicorn worker and can only deal with 1 request. But why would they not sequentially finish 20-50ms, one after the other? Our cpu doesn't really spike during this, and even giving more didn't help.
I'm sure 1 worker should be able to deal with 5 requests better than this. I feel like I'm missing something very obvious, or have misunderstood how multiple requests are handled.
Cheers for any points or guidance.
For what it's worth: I'm in the process of replacing our Auth requests with async requests, then flagging the views as async. Before moving on to updating our ORM operations to be Async - however this will take times and I'd like to understand better what I'm seeing today.
