Retrieve queue length with Celery (RabbitMQ, Django)

Question

I'm using Celery in a django project, my broker is RabbitMQ, and I want to retrieve the length of the queues. I went through the code of Celery but did not find the tool to do that. I found this issue on stackoverflow (Check RabbitMQ queue size from client), but I don't find it satisfying.

Everything is setup in celery, so there should be some kind of magic method to retrieve what I want, without specifying a channel / connection.

Does anyone have any idea about this question ?

Thanks !

mksm · Accepted Answer · 2017-05-24 02:29:34Z

4

Here is an example on how to read the queue length in rabbitMQ for a given queue:

def get_rabbitmq_queue_length(q):
    from pyrabbit.api import Client
    from pyrabbit.http import HTTPError

    count = 0

    try:
        cl = Client('localhost:15672', 'guest', 'guest')
        if cl.is_alive():
            count = cl.get_queue_depth('/', q)
    except HTTPError as e:
        print "Exception: Could not establish to rabbitmq http api: " + str(e) + " Check for port, proxy, username/pass configuration errors"
        raise

    return count

This is using pyrabbit as previously suggested by Philip

edited May 24, 2017 at 2:29

mksm

4,6275 gold badges41 silver badges57 bronze badges

answered Aug 30, 2016 at 14:20

Paul in 't Hout

4515 silver badges8 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Frederick Nord Over a year ago

so no way through celery as is?

Paul in 't Hout Over a year ago

@FrederickNord, not that I am aware of. My understanding is that the workers ( Celery ) is only concerned with it's current task, or at most the task it pre-fetched (from RabbitMQ). It's unaware of what is in RabbitMQ and intentionally so.

Philip Cristiano · Accepted Answer · 2013-07-26 11:14:05Z

3

PyRabbit is probably what you are looking for, it is a Python interface to the RabbitMQ management interface API. It will allow you to query for queues and their current message counts.

answered Jul 26, 2013 at 11:14

Philip Cristiano

9145 silver badges11 bronze badges

Comments

mksm · Accepted Answer · 2017-05-23 13:40:24Z

1

You can inspect the workers in celery by using inspect module. Here is the guide.

Also for RabbitMQ there are some command line command.

edited May 23, 2017 at 13:40

mksm

4,6275 gold badges41 silver badges57 bronze badges

answered Jul 25, 2013 at 16:53

Aamir Rind

39.8k24 gold badges131 silver badges169 bronze badges

3 Comments

user2619608 Over a year ago

thanks for your answer. But I don't need commands, I need python classes or methods. Plus, the inspect method will inspect the workers, not the queues, am I right ?

mksm Over a year ago

inspect can not retrieve the length of pending queue as I know. May be inspect reserved, but in my experience it doesn't help too..

Klaws Over a year ago

Indeed, the official Celery documentation states that you inspect the queues with the rabbitmqctl command line tool. That is pretty inconvenient when RabbitMQ runs on a different machine.

Collectives™ on Stack Overflow

Retrieve queue length with Celery (RabbitMQ, Django)

3 Answers 3

2 Comments

Comments

3 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

2 Comments

Comments

3 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related