How to implement multi-threading into a Python WSGI server

Question

Just to clarify up front that this is just a learning project and I have no intention of using this in production. There are several very good Python application servers out there already. But I am trying to learn more about concurrency, so I set out to write one of the things (I thought) I knew.

Also, because I wanted to "closer to the metal" so I started out with just Socket and want to keep it that way.

Below is the important parts of what I have so far. self.iq is a Queue object (inbound_queue) which then does nothing really, but puts the request (which includes the socket object) into the outbound_queue and then a Consumer object takes the request from the outbound_queue and passes it to the ResponseHandler. This seems to work fine with just me hitting it but I am concerned that I am opening myself up to a race condition with a naive implementation. Specifically assigning things to the ServerClass object that are request specific.

So the question is: Is there a better way to do this, or does my Queue usage prevent two threads from picking up and operating on the same object? Should I be encapsulating things like the WSGI environment stuff into a separate object that can be also passed into the queue? Doing stuff like that gets tricky in trying to write a server that is WSGI compliant because of the need to pass in callback functions.

class Consumer(threading.Thread):

    def __init__(self, out_queue, server):
        threading.Thread.__init__(self)
        self.out_queue = out_queue
        self.server = server

    def run(self):
        while True:
        item = self.out_queue.get()
        self.server.ResponseHandler(self.server, item)
        self.out_queue.task_done()

class QueueConsumerServer(object):

    methods_allowed = ['get', 'post', 'put', 'patch', 'delete', 'options', 'upgrade']

    def __init__(self, host, port, application):
        self.host = host
        self.port = port
        self.application = application
        self.iq = Queue.Queue()
        self.oq = Queue.Queue()

        self.socket = socket.socket()
        self.socket.bind((self.host, self.port))
        #<snip of lots of assigning stuff to environ>
        self.environ = environ
        headers_set = []
        headers_sent = []
        for i in xrange(3):
            thr = Producer(self.iq, self.oq)
            thr.daemon = True
            thr.start()

        for i in xrange(3):
            thr = Consumer(self.oq, self)
            thr.daemon = True
            thr.start()


    def handle_request(self):
        self.socket.listen(1)
        try:
            while True:
                cli, addr = self.socket.accept()
                data = cli.recv(1024)
                request_data = self.parse_request_data(data)
                req.path = request_data[1]
                req.cli = cli
                self.iq.put(req)
                return
        except Exception, ex:
            print 'e', ex,
            sys.exit(1)
        finally:
            sys.stdout.flush()
            self.socket.close()

not true. a closer to the metal wsgi implementation is actually faster! — user3717756
– user3717756, Commented Jul 5, 2014 at 14:49

cmh · Accepted Answer · 2013-06-03 16:28:30Z

2

Python queues are thread safe, so there is no race in your code as written.

Regarding a better way to approach this, your code will probably hit the GIL before too long. I would suggest looking to into multiprocessing.

answered Jun 3, 2013 at 16:28

cmh

11k5 gold badges33 silver badges40 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

zenWeasel Over a year ago

Yes, I am aware of the multiprocessing module. This was a learning exercise, so I am running through all of the different types of concurrency. Thread, Multiprocessing, Greenlets, Coroutines, etc.

cmh Over a year ago

Good stuff. It's worth bearing in mind that threads in python have inferior performance characteristics to threads in other languages (e.g. c++, java), due to the GIL. They are still useful for IO multiplexing, but you're going to need to use processes if you want to exploit multiple cores in vanilla python.

zenWeasel Over a year ago

Understood. I can't imagine using this for doing anything except maybe a static file server where you are IO-bound. But as I study threading/concurrency, web servers tend to be my "hello world". Which is actually all this server serves right now. Thanks for the quick answer.

Collectives™ on Stack Overflow

How to implement multi-threading into a Python WSGI server

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related