ListField does not respect ordered sequence in form data with partial updates.

[proofit404]

Checklist

• I have verified that that issue exists against the master branch of Django REST framework.
• I have searched for similar issues in both open and closed tickets and cannot find a duplicate.
• This is not a usage question. (Those should be directed to the discussion group instead.)
• This cannot be dealt with as a third party library. (We prefer new functionality to be in the form of third-party libraries where possible.)
• I have reduced the issue to the simplest possible case.
• I have included a failing test as a pull request. (If you are unable to do so we can still accept the issue.)

Steps to reproduce

1. For example, we have serializer like this:

class CommunitySerializer(Serializer):

    colors = ListField(allow_null=True, child=CharField(label='Colors', max_length=7), required=False)

2. We submit partial update encoded as form data:

test_client.patch('/communities/1/', {'colors[0]': '#ffffff', 'colors[1]': '#000000'}, 'multipart')

3. This will encode request body like this:

--BoUnDaRyStRiNg
Content-Disposition: form-data; name="colors[0]"

#000000
--BoUnDaRyStRiNg
Content-Disposition: form-data; name="colors[1]"

#ffffff
--BoUnDaRyStRiNg--

Expected behavior

validated_data should contain colors value parsed into a list with the respected order.

Actual behavior

validated_data is an empty dictionary.

Explanation

1. ListField.get_value checks that field is present with in statement:

django-rest-framework/rest_framework/fields.py

Line 1639 in ed6340e

if self.field_name not in dictionary:

2. Since field_name does not match submitted keys, method checks in this is a partial update

django-rest-framework/rest_framework/fields.py

Line 1640 in ed6340e

if getattr(self.root, 'partial', False):

3. Since this is also true, the method returns an empty marker.

django-rest-framework/rest_framework/fields.py

Line 1641 in ed6340e

return empty

4. Field name with ordered marker does not check to present.

Alternative

We can submit the partial update with list fields unordered:

test_client.patch('/communities/1/', {'colors': ['#ffffff', '#000000']}, 'multipart')

Form data will look like this

--BoUnDaRyStRiNg
Content-Disposition: form-data; name="colors"

#000000
--BoUnDaRyStRiNg
Content-Disposition: form-data; name="colors"

#ffffff
--BoUnDaRyStRiNg--

So in statement will evaluate to true, so dictionary will be processed by HTML parser.

django-rest-framework/rest_framework/fields.py

Line 1649 in ed6340e

return html.parse_html_list(dictionary, prefix=self.field_name, default=empty)

But this format does not guarantee items order in the list, so we can not use it for things like gradients.

Full update and create methods do not have this problem because partial check evaluates to false.

We can't use full updates because we have two file fields near. We don't want to force the user to upload two files just to change string values in the same request.

P.S.

If you think we should solve this, I'll be able to make a PR for it.

[rawteech]

I don't know if you still having an issue with this but try changing

class CommunitySerializer(Serializer):
    colors = ListField(allow_null=True, child=CharField(label='Colors', max_length=7), required=False)

to

class CommunitySerializer(Serializer):
    colors = ListField(allow_empty=True, child=CharField(label='Colors', required=False), max_length=7)

[jttttttttttttt]

version:3.11.0
I have the same problem. When I use http post , the data {'colors[0]': '#ffffff', 'colors[1]': '#000000'} is OK, but use http patch get problem, there is no data, only use {'colors': '#ffffff'} get colors=["'#ffffff'"]

[Megajoe17]

Have same problem.
Checked for version djangorestframework-3.10.3 and djangorestframework-3.12.2
Still working incorrect.
Solved with nested class from ListField

class ListFieldExt(ListField):
    # this class fixes problem when PATCH with indexed params {'a[0]': 1, 'a[1]': 2}
    def get_value(self, dictionary):
        if self.field_name not in dictionary:
            if getattr(self.root, 'partial', False):
                # fix PATCH method problem
                parsed_list = html.parse_html_list(dictionary, prefix=self.field_name, default=empty)
                if parsed_list != empty:
                    return parsed_list

                return empty
        # We override the default field access in order to support
        # lists in HTML forms.
        if html.is_html_input(dictionary):
            val = dictionary.getlist(self.field_name, [])
            if len(val) > 0:
                # Support QueryDict lists in HTML input.
                return val
            return html.parse_html_list(dictionary, prefix=self.field_name, default=empty)

        return dictionary.get(self.field_name, empty)

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[auvipy]

I don't know if you still having an issue with this but try changing

class CommunitySerializer(Serializer):
    colors = ListField(allow_null=True, child=CharField(label='Colors', max_length=7), required=False)

to

class CommunitySerializer(Serializer):
    colors = ListField(allow_empty=True, child=CharField(label='Colors', required=False), max_length=7)

can we close this based on what you suggested?

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[Natgho]

#9902