Django Views: When is request.data a dict vs a QueryDict?

Question

I have run into some trouble with the issue, that request.data sometimes is a dict (especially when testing) and sometimes a QueryDict instance (when using curl).

This is especially a problem because apparently there is a big difference when calling a view using curl like so:

curl -X POST --data "some_float=1.23456789012123123" "http://localhost:8000/myview"

Or using the django_webtest client like so:

class APIViewTest(WebTest):
    def test_testsomething(self):
        self.app.post(url=url, params=json.dumps({some_float=1.26356756467}))

And then casting that QueryDict to a dict like so

new_dict = dict(**request.data)
my_float = float(new_dict['some_float'])

Everything works fine in the tests, as there request.data is a dict, but in production the view crashes because new_dict['some_float'] is actually a list with one element, and not as expected a float.

I have considered fixing the issue like so:

    if type(request.data) is dict:
        new_dict = dict(**request.data)
    else:
        new_dict = dict(**request.data.dict())

which feels very wrong as the tests would only test line 2, and (some? all?) production code would run line 4.

So while I am wondering why QueryDict behaves in this way, I would rather know why and when response.data is a QueryDict in the first place. And how I can use django tests to simulate this behavior. Having different conditions for production and testing systems is always troublesome and sometimes unavoidable, but in this case I feel like it could be fixed. Or is this a specific issue related to django_webtest?

Kennie

Because you can pass two values for the same key, like: curl -X POST --data "some_float=1.23456789012123123" --data "some_float=3.14" "http://localhost:8000/myview". This in essence why they built a QueryDict in the first place. – Poetize

But shouldn't request.data then always be a QueryDict instance? – Kennie

It is, it is only because you here convert it to adict, that it is, of course, a dictionary. – Poetize

No it is not. type(request.data) is dict returns True when testing locally as explained in my question. – Kennie

Answer 1

When your request content_type is "application/x-www-form-urlencoded", request.Data become QueryDict.

see FormParser class.
https://github.com/encode/django-rest-framework/blob/master/rest_framework/parsers.py

And

QueryDict has get lists method. but it can't fetch dict value.
convert name str to array.

<input name="items[name]" value="Example">
<input name="items[count]" value="5">  

https://pypi.org/project/html-json-forms/

And define custom form paser.

class CustomFormParser(FormParser):
"""
Parser for form data.
"""
media_type = 'application/x-www-form-urlencoded'

def parse(self, stream, media_type=None, parser_context=None):
    """
    Parses the incoming bytestream as a URL encoded form,
    and returns the resulting QueryDict.
    """
    parser_context = parser_context or {}
    encoding = parser_context.get('encoding', settings.DEFAULT_CHARSET)
    data = QueryDict(stream.read(), encoding=encoding)
    return parse_json_form(data.dict()) # return dict

And overwite DEFAULT_PARSER_CLASSES.
https://www.django-rest-framework.org/api-guide/settings/#default_parser_classes

Four

Answer 2

Your test isn't a reflection of your actual curl call.

In your test, you post JSON, which is then available as a dict from request.data. But your curl call posts standard form data, which is available as a QueryDict. This behaviour is managed by the parsers attribute of your view or the DEFAULT_PARSER_CLASSES settings - and further note that this is functionality specifically provided by django-rest-framework, not Django itself.

Really you should test the same thing as you are doing; either send JSON from curl or get your test to post form-data.

Starspangled

Unfortunately my clients send both. I hoped django had some sort of interface which could deal with both in the same way – Kennie

But as I just said, that is DRF functionality which is controlled by the parsers setting, you are free to override this or set your own. In any case, the question "when is request.data a dict vs a querydict" is answered by "when you send JSON vs form-data". – Starspangled

And additionally, request.data is the interface that deals with both in the same way. I don't know why you think you need to convert to a dict at all, a QueryDict operates exactly like a dict in all respects other than it can additionally handle multiple values for the same key. If you don't have multiple values, just use it like a dict and you won't notice any difference. – Starspangled

It does not behave exactly the same way. One is mutable, the other is not. – Kennie

I wish this answer was more explicit regarding how to fix the issue, rather than just pointing it out – Semiquaver