Handling request with chunked transfer-encoding

Question

Does golang's net/http package support requests with chunked transfer-encoding? Thus far I have been able to use the Hijacker interface (https://golang.org/src/net/http/server.go?s=6173:6875#L156) to at least not close the connection and receive the full chunked request, but not yet parsing the chunks and suspect I may be going down the wrong path with this.

From https://golang.org/src/net/http/httputil/httputil.go?s=688:732#L10, I see there is a chunked reader, but appears to be for internal use.

Essentially, I'm trying to accept an HTTP PUT with 'chunked' transfer-encoding and send it off to a backend server 'on-the-fly' (i.e. without buffering the full request in golang). I have no control over the upstream request. Is there a recommended way to handle such a request, or is Hijacker the way to do it?

score 6 · Accepted Answer · 2017-12-22 21:15:54Z

6

The net/http client and server transparently read and write chunked bodies.

To accept a chunked request and send it to another HTTP server, pass the server request body as the client request body. Here's now to forward the body to another server as a PUT:

 func handler(w http.ResponseWriter, r *http.Request) {
    creq, err := http.NewRequest("PUT", url, r.Body)
    if err != nil {
       // handle error
    }
    if ct := r.Header.Get("Content-Type"); ct != "" {
        creq.Header.Set("Content-Type", ct)
    }
    cresp, err := http.DefaultClient.Do(creq)
    if err != nil {
        // handle error
    }
    ... do something with cresp.
 }

If you want to copy to a file, then io.Copy the request body to the file.

 func handler(w http.ResponseWriter, r *http.Request) {
     f, err := os.Create("fname")
     if err != nil {
         // handle error
     }
     _, err := io.Copy(f, r.Body)
     if err != nil {
          // handle error
     }
     ...
  }

These snippets copy the body 'on the fly'.

edited Dec 22, 2017 at 21:15

answered Dec 22, 2017 at 20:34

user5728991

Sign up to request clarification or add additional context in comments.

1 Comment

thinkski Over a year ago

Oh wow, this is much simpler -- thank you! And if initially I just wanted to write to a file?

Collectives™ on Stack Overflow

Handling request with chunked transfer-encoding

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related