Django Rest Framework does not decode JSON fields in Multipart requests like it does with standard POST requests

Question

When I'm submitting a POST request with a Content-Type of application/json, my data at the server is decoded into native Python as it should - JSON objects are presented as dicts, arrays as arrays etc. That's great.
However, when I'm doing a MultiPart post request to my API, which of course also contains a file, any field containing JSON/objects is not decoded at the server and I'm left with strings I need to decode myself. The nature of my app means that I cannot always know which fields I'm about to get.

My question is - how can I submit a multipart request with a file and yet retain DRF's ability to decode JSON data in some of the fields? I've tried using all 3 major parsers together but that didn't work (by setting the API view's parser_classes to them: parser_classes = (MultiPartParser, JSONParser, FormParser)

Here are some example requests sent (via Chrome's dev tools):

Standard Post (not multipart, no file): {"first_name":"Test","last_name":"Shmest","email":[{"channel_type":"Email","value":"[email protected]","name":null,"default":false}],"company":{"position":"Manager","id":"735d2b5f-e032-4ca8-93e4-c7773872d0cc","name":"The Compapa"},"access":{"private":true,"users":[10,1]},"description":"Nice guy!!","address":{"city":"San Fanfanfo","zip":"39292","country":"United States of America","state":"CA","map_url":null,"country_code":"US","address":"123 This street"},"phone":[{"default":false,"type":"Phone","id":"70e2b437-6841-4536-9acf-f6a55cc372f6","value":"+141512312345","name":null}],"position":"","department":"","supervisor":"","assistant":"","referred_by":"","status":"","source":"","category":"Lead","do_not_call":false,"do_not_text":false,"do_not_email":false,"birthday":null,"identifier":""} This payload gets read by DRF just fine, and all values are set to their native equivalent.

Multipart with file and one of the fields is JSON encoded object:

------WebKitFormBoundaryPfKUrmBd9vRwp5Rb
Content-Disposition: form-data; name="file"; filename="image.png"
Content-Type: image/png


------WebKitFormBoundaryPfKUrmBd9vRwp5Rb
Content-Disposition: form-data; name="first_name"

Test
------WebKitFormBoundaryPfKUrmBd9vRwp5Rb
Content-Disposition: form-data; name="last_name"

Shmest
------WebKitFormBoundaryPfKUrmBd9vRwp5Rb
Content-Disposition: form-data; name="email"

[{"channel_type":"Email","value":"[email protected]","name":null,"default":false}]
------WebKitFormBoundaryPfKUrmBd9vRwp5Rb
Content-Disposition: form-data; name="company"

{"position":"Manager","id":"735d2b5f-e032-4ca8-93e4-c7773872d0cc","name":"The Compapa"}
------WebKitFormBoundaryPfKUrmBd9vRwp5Rb
Content-Disposition: form-data; name="access"

{"private":true,"users":[10,1]}
------WebKitFormBoundaryPfKUrmBd9vRwp5Rb
Content-Disposition: form-data; name="description"

Nice guy!!
------WebKitFormBoundaryPfKUrmBd9vRwp5Rb
Content-Disposition: form-data; name="address"

What I'm looking for is to see if there is some way to have the JSON decoding be automatic in multipart request like it is in regular POST, before I dive into manually decoding all fields to check if they are JSON or not. Most fields I'll be getting will be unknown to me until the request is made as each user might have a different combination of fields.

Do you have any control of the client data that has been send? How do you not know, which fields is being send (that implicates that even using the json parser would not work either..)? — mariodev
– mariodev, Commented Aug 6, 2016 at 6:05
Yes I do. It's a react app, using superagent for api calls. In multipart requests (not many, just 2), I had to stringify the data myself but it's still valid json. — Harel
– Harel, Commented Aug 6, 2016 at 9:09
If you know that the data is always gonna be json data + file format, then you can simply decode the json in serializer or view. Unless you're use this flow in other views there's no need to write custom parser, since the json parser does exactly the same - it just decodes json.. — mariodev
– mariodev, Commented Aug 6, 2016 at 9:50
The problem is that the incoming fields are not consistent. The whole system is dynamic and anything can come in (type wise : some strings, some numbers, some complex json). The system is mostly ready and only now I notice the problem when sending multipart requests with files. I thought of breaking uploads to separate endpoint and decode myself the one case I can't but still it's something I rather solve as a whole and do it right so it doesn't matter if it's multipart or regular post. — Harel
– Harel, Commented Aug 6, 2016 at 10:00

Harel · Accepted Answer · 2016-08-06 20:54:00Z

I created a new Parser object to deal with this scenario of MultiPart file upload with fields containing JSON. Code is below if anyone ever needs it.

import json
from rest_framework.parsers import BaseParser, DataAndFiles
from django.conf import settings
from django.http.multipartparser import MultiPartParser as DjangoMultiPartParser, MultiPartParserError
from django.utils import six
from rest_framework.exceptions import ParseError


class MultiPartJSONParser(BaseParser):
    """
    Parser for multipart form data which might contain JSON values
    in some fields as well as file data.
    This is a variation of MultiPartJSONParser, which goes through submitted fields
    and attempts to decode them as JSON where a value exists. It is not to be used as a replacement
    for MultiPartParser, only in cases where MultiPart AND JSON data are expected.
    """
    media_type = 'multipart/form-data'

    def parse(self, stream, media_type=None, parser_context=None):
        """
        Parses the incoming bytestream as a multipart encoded form,
        and returns a DataAndFiles object.
        `.data` will be a `QueryDict` containing all the form parameters, and JSON decoded where available.
        `.files` will be a `QueryDict` containing all the form files.
        """
        parser_context = parser_context or {}
        request = parser_context['request']
        encoding = parser_context.get('encoding', settings.DEFAULT_CHARSET)
        meta = request.META.copy()
        meta['CONTENT_TYPE'] = media_type
        upload_handlers = request.upload_handlers

        try:
            parser = DjangoMultiPartParser(meta, stream, upload_handlers, encoding)
            data, files = parser.parse()
            for key in data:
                if data[key]:
                    try:
                        data[key] = json.loads(data[key])
                    except ValueError:
                        pass
            return DataAndFiles(data, files)
        except MultiPartParserError as exc:
            raise ParseError('Multipart form parse error - %s' % six.text_type(exc))

The parser can be used within the API view like any other:

parser_classes = (MultiPartJSONParser, JSONParser , FormParser)

Collectives™ on Stack Overflow

Django Rest Framework does not decode JSON fields in Multipart requests like it does with standard POST requests

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related