approximate count of rows in postgresql django

Question

I have a postgresql database connected to with django. In database there is lots of big tables which I want their row count. Because of large size of tables this takes a lot of time to execute.

I founded that the approximate count could be retrieved from pg_class. Is there any way to this in Django and not executing raw query?

You can do this with a FuzzyCountManager: github.com/stephenmcd/django-postgres-fuzzycount — willeM_ Van Onsem
– willeM_ Van Onsem, Commented Oct 20, 2018 at 11:22

willeM_ Van Onsem · Accepted Answer · 2021-03-03 12:22:40Z

3

I propose to use a dedicated package for this named django-postgres-fuzzycount [GitHub]. This package provides a manager that will do the fast counting.

You can install the package with pip (for example in your local environment):

$ pip install django-postgres-fuzzycount

Then you can add the FuzzyCountManager to the models where you want to obtain an approximative count:

from django.db import models
from fuzzycount import FuzzyCountManager

class SomeModel(models.Model):

    #  ... (some fields) ...

    objects = models.Manager()
    approx = FuzzyCountManager()

and then you can count approximatively with:

SomeModel.approx.count()

In case you .filter(..), Django will calculate the real number, since the pg_class table only stores an apprixmative number of rows for the entire table, so:

SomeModel.approx.filter(foo=bar).count()

will take more time (depending on indexes, etc.).

You can also "patch" the objects manager directly, but then obtaining the real number of records might be harder:

from django.db import models
from fuzzycount import FuzzyCountManager

class SomeModel(models.Model):

    #  ... (some fields) ...

    objects = FuzzyCountManager()

It is also nice that if you change the backend database to another database, the FuzzyCountManager(..) will act as a normal Manager, so in case you later change the database system, you do not have to rewrite the managers.

edited Mar 3, 2021 at 12:22

answered Oct 20, 2018 at 12:24

willeM_ Van Onsem

482k33 gold badges483 silver badges624 bronze badges

Sign up to request clarification or add additional context in comments.

10 Comments

Miguel Over a year ago

I don't want to "patch" the objects directly, because already have a custom manager, Added approx = FuzzyCountManager() but got the error: AttributeError: 'QuerySet' object has no attribute 'approx'

willeM_ Van Onsem Over a year ago

@Miguel: you are likely working with SomeModel.objects.approx, but then it is a QuerySet, so that will not work. Note that .objects is only defined on the model, not on a manager or QuerySet that arises from that model.

willeM_ Van Onsem Over a year ago

@Miguel: you can however simply make a manager in the model approx = FuzzyCountManager(), and another ojbects = models.Manager(), then you thus can work with SomeModel.approx to count the number of elements approximately.

Miguel Over a year ago

Ok, thanks. But what about count the queryset? That is what I need.

willeM_ Van Onsem Over a year ago

@Miguel: you can count exactly with SomeModel.objects.count() or approximately with SomeModel.approx.count() if you implement it as in the first way (second code fragment).

|

Collectives™ on Stack Overflow

approximate count of rows in postgresql django

1 Answer 1

10 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

10 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related