-1

I am getting a weird error after upgrading the Django version from 4.0 to 4.2.15. The error being encountered is: RecursionError: maximum recursion depth exceeded while calling a Python object.

The ModelDiffMixin looks something like below:

from django.forms.models import model_to_dict

class ModelDiffMixin(object):

def __init__(self, *args, **kwargs):
    super(ModelDiffMixin, self).__init__(*args, **kwargs)
    self.__initial = self._dict

@property
def diff(self):
    d1 = self.__initial
    d2 = self._dict
    diffs = [(k, (v, d2[k])) for k, v in d1.items() if v != d2[k]]
    return dict(diffs)

@property
def has_changed(self):
    return bool(self.diff)

@property
def changed_fields(self):
    return self.diff.keys()

def get_field_diff(self, field_name):
    """
    Returns a diff for field if it's changed and None otherwise.
    """
    return self.diff.get(field_name, None)

def save(self, *args, **kwargs):
    """
    Saves model and set initial state.
    """
    super(ModelDiffMixin, self).save(*args, **kwargs)
    self.__initial = self._dict

@property
def _dict(self):
    return model_to_dict(self, fields=[field.name for field in
                         self._meta.fields])

Referenced from the gist here: https://gist.github.com/goloveychuk/72499a7251e070742f00

Attaching the stack trace here. I have gone through the django docs as well and it seems like accessing django fields in the init method of model might cause issues.

  File "/Users/bm/.virtualenvs/django4.2/lib/python3.11/site-packages/django/db/models/query.py", line 122, in __iter__
    obj = model_cls.from_db(
          ^^^^^^^^^^^^^^^^^^
  File "/Users/bm/.virtualenvs/django4.2/lib/python3.11/site-packages/django/db/models/base.py", line 582, in from_db
    new = cls(*values)
          ^^^^^^^^^^^^
  File "/Users/bm/.virtualenvs/django4.2/lib/python3.11/site-packages/django/db/models/base.py", line 571, in __init__
    super().__init__()
  File "/Users/bm/Desktop/Django/bapi/core/utils.py", line 2146, in __init__
    self.__initial = self._dict
                     ^^^^^^^^^^
  File "/Users/bm/Desktop/Django/bapi/core/utils.py", line 2201, in _dict
    return model_to_dict(self, fields=[field.name for field in self._meta.fields])
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/bm/.virtualenvs/django4.2/lib/python3.11/site-packages/django/forms/models.py", line 115, in model_to_dict
    data[f.name] = f.value_from_object(instance)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/bm/.virtualenvs/django4.2/lib/python3.11/site-packages/django/db/models/fields/__init__.py", line 1088, in value_from_object
    return getattr(obj, self.attname)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/bm/.virtualenvs/django4.2/lib/python3.11/site-packages/django/db/models/query_utils.py", line 178, in __get__
    instance.refresh_from_db(fields=[field_name])
  File "/Users/bm/.virtualenvs/django4.2/lib/python3.11/site-packages/django/db/models/base.py", line 724, in refresh_from_db
    db_instance = db_instance_qs.get()
                  ^^^^^^^^^^^^^^^^^^^^
  File "/Users/bm/.virtualenvs/django4.2/lib/python3.11/site-packages/django/db/models/query.py", line 633, in get
    num = len(clone)
          ^^^^^^^^^^
  File "/Users/bm/.virtualenvs/django4.2/lib/python3.11/site-packages/django/db/models/query.py", line 380, in __len__
    self._fetch_all()
  File "/Users/bm/.virtualenvs/django4.2/lib/python3.11/site-packages/django/db/models/query.py", line 1881, in _fetch_all
    self._result_cache = list(self._iterable_class(self))
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/bm/.virtualenvs/django4.2/lib/python3.11/site-packages/django/db/models/query.py", line 91, in __iter__
    results = compiler.execute_sql(
              ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/bm/.virtualenvs/django4.2/lib/python3.11/site-packages/django/db/models/sql/compiler.py", line 1547, in execute_sql
    sql, params = self.as_sql()
                  ^^^^^^^^^^^^^
  File "/Users/bm/.virtualenvs/django4.2/lib/python3.11/site-packages/psqlextra/compiler.py", line 76, in as_sql
    sql, params = super().as_sql(*args, **kwargs)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/bm/.virtualenvs/django4.2/lib/python3.11/site-packages/django/db/models/sql/compiler.py", line 734, in as_sql
    extra_select, order_by, group_by = self.pre_sql_setup(
                                       ^^^^^^^^^^^^^^^^^^^
  File "/Users/bm/.virtualenvs/django4.2/lib/python3.11/site-packages/django/db/models/sql/compiler.py", line 85, in pre_sql_setup
    order_by = self.get_order_by()
               ^^^^^^^^^^^^^^^^^^^
  File "/Users/bm/.virtualenvs/django4.2/lib/python3.11/site-packages/django/db/models/sql/compiler.py", line 457, in get_order_by
    for expr, is_ref in self._order_by_pairs():
  File "/Users/bm/.virtualenvs/django4.2/lib/python3.11/site-packages/django/db/models/sql/compiler.py", line 339, in _order_by_pairs
    selected_exprs[expr] = pos_expr
    ~~~~~~~~~~~~~~^^^^^^
  File "/Users/bm/.virtualenvs/django4.2/lib/python3.11/site-packages/django/db/models/expressions.py", line 502, in __hash__
    return hash(self.identity)
                ^^^^^^^^^^^^^
  File "/Users/bm/.virtualenvs/django4.2/lib/python3.11/site-packages/django/utils/functional.py", line 57, in __get__
    res = instance.__dict__[self.name] = self.func(instance)
                                         ^^^^^^^^^^^^^^^^^^^
  File "/Users/bm/.virtualenvs/django4.2/lib/python3.11/site-packages/django/db/models/expressions.py", line 479, in identity
    constructor_signature = inspect.signature(self.__init__)
                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/[email protected]/3.11.9/Frameworks/Python.framework/Versions/3.11/lib/python3.11/inspect.py", line 3263, in signature
    return Signature.from_callable(obj, follow_wrapped=follow_wrapped,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/[email protected]/3.11.9/Frameworks/Python.framework/Versions/3.11/lib/python3.11/inspect.py", line 3011, in from_callable
    return _signature_from_callable(obj, sigcls=cls,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/[email protected]/3.11.9/Frameworks/Python.framework/Versions/3.11/lib/python3.11/inspect.py", line 2447, in _signature_from_callable
    _get_signature_of = functools.partial(_signature_from_callable,
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    RecursionError: maximum recursion depth exceeded while calling a Python object
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Accessing in Django models like:

class Member(ModelDiffMixin):
   class Meta:
    index_together = [['facebook_id'], ['enrollment_referrer']]
    indexes = [models.Index(fields=['-created_ts'])]

   member_name = models.CharField(max_length=100, null=True, blank=True)
   middle_name = models.CharField(max_length=100, null=True, blank=True)

There is no change made in the code and the same code seems to work with Django 4.0 but not with Django 4.2.15. Would really appreciate if someone can help out with this.

---- Edited ------

Figured out the actual cause. The stack trace was misleading, and debugged bit deeper to find the piece of code that is making this happen. The error is because of serializing model instance:

from django.core import serializers
def get_serialized_instance(instance):
    data = serializers.serialize('json', [instance])
    result = json.loads(data)[0].get('fields')
    result['id'] = instance.id
    return result

This piece of code is the actual cause of recursion. It works with Django 4.0 but not with Django 4.2.15. Wonder if Django has dropped supporting serializing model instances.

Improve this question
6
  • Is this complete model definition? Do you have any ordering defined on it? Commented Sep 23, 2024 at 10:48
  • @KrzysztofSzularz No ordering field defined. The model definition is not complete, it has around 50+ fields, didn't mention them as it would have added a lot of code. Commented Sep 23, 2024 at 10:54
  • why use _dict property?? Commented Sep 25, 2024 at 1:49
  • Please provide a proper minimal reproducible example that we can run as is to reproduce your issue. What code are you calling to get the error? It looks like you fired some sort of query and it got an error while trying to create the result objects. Also class Member(ModelDiffMixin) is not even a Django model so you are very obviously missing a minimal reproducible example since we won't be able to fire off a query with it... Commented Sep 25, 2024 at 5:19
  • You don't need the self.__initial = self._dict in the __init__(), which is where you're getting the error from. It would make sense to need to check for changes only after a save. Commented Sep 26, 2024 at 17:28

2 Answers 2

Reset to default
1

Description

Just like as Ben said it in the comments, self.__inital = self._dict in the init() function is causing the recursion as _dict hasn't been initialized yet. One way to do this is to put it into a save() method.

Code

Maybe try doing something like this:

from django.forms.models import model_to_dict

class ModelDiffMixin(object):

    def __init__(self, *args, **kwargs):
        super(ModelDiffMixin, self).__init__(*args, **kwargs)
        self.__initial = None

    def save(self, *args, **kwargs):
        super(ModelDiffMixin, self).save(*args, **kwargs)
        self.__initial = self._dict

    @property
    def diff(self):
        if self.__initial is None:
            return {}
        d1 = self.__initial
        d2 = self._dict
        diffs = [(k, (v, d2[k])) for k, v in d1.items() if v != d2[k]]
        return dict(diffs)

    @property
    def has_changed(self):
        return bool(self.diff)

    @property
    def changed_fields(self):
        return self.diff.keys()

    def get_field_diff(self, field_name):
        """
        Returns a diff for field if it's changed and None otherwise.
        """
        return self.diff.get(field_name, None)

    @property
    def _dict(self):
        return model_to_dict(self, fields=[field.name for field in self._meta.fields])

    def refresh_initial_state(self):
        self.__initial = self._dict

Measures:

Do take your time to refine the code, it will not be perfect as I don't know what the rest of the code is. I can take a better look if you can share a working example.

Improve this answer
Sign up to request clarification or add additional context in comments.

Comments

0

Figured out the issue:

The dependency between two models caused the recursion issue.

class Offer(A, ModelDiffMixin):
   offer_name = models.CharField(max_length=50)
   locations = models.ManyToManyField(SponsorLocation)

class SponsorLocation(B, ModelDiffMixin):
   sponsor = models.ForeignKey(Sponsor, on_delete=models.CASCADE)

When trying to serialize Offer model, the issue occured because of this particular line of code:

data = serializers.serialize('json', [instance])

ModelDiffMixin might be trying to track changes in the locations field by traversing the ManyToMany relationship between Offer and SponsorLocation. As it checks the changes in Offer, it is likely navigating to SponsorLocation, and since SponsorLocation also has ModelDiffMixin, it might be traversing back to Offer, causing a recursive loop.

The solution is sort of a hack, where the locations field is excluded from the serialization process and then appended later in the result.

def get_serialized_instance(instance):
    if type(instance) == Offer:
        fields = [
            field.name
            for field in instance._meta.get_fields()
            if field.name not in ['locations']
        ]
        data = serializers.serialize('json', [instance], fields=fields)
        result = json.loads(data)[0].get('fields')
        # Serialize the locations field separately for Offer model
        result['locations'] = list(instance.locations.values_list('id', flat=True))
    else:
        data = serializers.serialize('json', [instance])
        result = json.loads(data)[0].get('fields')
    result['id'] = instance.id
    return result
Improve this answer

4 Comments

Where is the solution? What did you change to make it work?
This snippet isn't even present in the question 😂
@Ben Added the hack i did to make it work.
@AshhadDevLab The serialization part is already added in the question posted. I did mention that the stack trace was misleading since it did not match the actual cause of the issue.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.