How to parse an HTTP request in python (custom web server)

Question

I am trying to create a multi-threaded web server in python. I have managed to get it to work with basic text, but I'm having trouble adapting it to HTTP.

Also it crashes when sending a GET request, but it does not crash when just using 'connect'. Am I not setting up the server correctly on localhost:port ?

The main problem however, is that I have a client socket, but I do not know how to extract the request data from it.

#! /usr/bin/env python3.3
import socket, threading, time, sys, queue

#initial number of worker threads
kNumThreads = 50

#queue of all the incoming requests
connections = queue.Queue(-1)


class Request_Handler(threading.Thread):
    def __init__(self):
        threading.Thread.__init__(self)

    def run(self):
        while(True):
            #Get the request socket data from the queue
            global connections
            self.connection = connections.get()
            self.addr = self.connection[1]
            self.socket = self.connection[0]

            #I have the socket, but how do I parse the request from here?
            #Also is there a better way to respond than to manually create a response here?

            self.response = 'HTTP/1.1 200 OK/nContent-Type: text/html; charset=UTF-8/n/n <html><body>Hello World</body></html>'
            print("got here")
            self.socket.send(self.response.encode('utf-8'))
            self.socket.close()
            requests.task_done()


#creates kNumThreads threads that will handle requests
def create_threads():
    for n in range(0, kNumThreads):
       RH = Request_Handler()
       RH.start()

def main():
    s = socket.socket()
    port = int(sys.argv[1])

    #sets up the server locally
    s.bind(('127.0.0.1', port))
    s.listen(100)

    #create the worker threads   
    create_threads()

    #accept connections and add them to queue
    while(True):
        c, addr = s.accept()
        connections.put_nowait((c, addr))

if __name__ == '__main__':
    main()

Yes but I believe the requests module is only for making requests, and not for parsing them — Mariano Sorgente
– Mariano Sorgente, Commented Sep 4, 2013 at 6:12
@SrinivasReddyThatiparthy. Chapeau, really. It is very rare to find people who -without getting angry- accept corrections and criticisms. Especially on a forum. Chapeau. — Hyperboreus
– Hyperboreus, Commented Sep 4, 2013 at 6:24
@Hyperboreus Apologies. My bad english. I thought the meaning was reverse.Just now googled and found it. — Srinivas Reddy Thatiparthy
– Srinivas Reddy Thatiparthy, Commented Sep 4, 2013 at 6:26

tuxcanfly · Accepted Answer · 2013-09-04 06:44:52Z

1

You need to receive data from the connection like this:

while(True):
    c, addr = s.accept()
    data = c.recv(1024)
    print(data)
    connections.put_nowait((c, addr))

Refer to the HTTP spec on how to read the request

http://www.w3.org/Protocols/rfc2616/rfc2616-sec4.html#sec4.4

answered Sep 4, 2013 at 6:44

tuxcanfly

2,5841 gold badge20 silver badges18 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Mariano Sorgente Over a year ago

Alright I guess I can manually parse it, but I was wondering if there was a good library for this

Collectives™ on Stack Overflow

How to parse an HTTP request in python (custom web server)

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related