when using laravel queue job processing, getting stuck

Question

i am on 5.3.31

so it cannot be related to

https://github.com/laravel/framework/issues/15179

after 300ish jobs I get:

[2017-04-11 13:51:53] local.ERROR: Symfony\Component\Debug\Exception\FatalThrowableError: Call to a member function beginTransaction() on null in /var/www/html/www.myapp.com/vendor/laravel/framework/src/Illuminate/Database/Connection.php:612
Stack trace:
#0 /var/www/html/www.myapp.com/vendor/laravel/framework/src/Illuminate/Queue/DatabaseQueue.php(175): Illuminate\Database\Connection->beginTransaction()
#1 /var/www/html/www.myapp.com/vendor/laravel/framework/src/Illuminate/Queue/Worker.php(175): Illuminate\Queue\DatabaseQueue->pop('default')
#2 /var/www/html/www.myapp.com/vendor/laravel/framework/src/Illuminate/Queue/Worker.php(145): Illuminate\Queue\Worker->getNextJob(Object(Illuminate\Queue\DatabaseQueue), 'default')
#3 /var/www/html/www.myapp.com/vendor/laravel/framework/src/Illuminate/Queue/Worker.php(75): Illuminate\Queue\Worker->runNextJob('database', 'default', Object(Illuminate\Queue\WorkerOptions))
#4 /var/www/html/www.myapp.com/vendor/laravel/framework/src/Illuminate/Queue/Console/WorkCommand.php(100): Illuminate\Queue\Worker->daemon('database', 'default', Object(Illuminate\Queue\WorkerOptions))
#5 /var/www/html/www.myapp.com/vendor/laravel/framework/src/Illuminate/Queue/Console/WorkCommand.php(83): Illuminate\Queue\Console\WorkCommand->runWorker('database', 'default')
#6 [internal function]: Illuminate\Queue\Console\WorkCommand->fire()
#7 /var/www/html/www.myapp.com/vendor/laravel/framework/src/Illuminate/Container/Container.php(508): call_user_func_array(Array, Array)
#8 /var/www/html/www.myapp.com/vendor/laravel/framework/src/Illuminate/Console/Command.php(169): Illuminate\Container\Container->call(Array)
#9 /var/www/html/www.myapp.com/vendor/symfony/console/Command/Command.php(256): Illuminate\Console\Command->execute(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#10 /var/www/html/www.myapp.com/vendor/laravel/framework/src/Illuminate/Console/Command.php(155): Symfony\Component\Console\Command\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#11 /var/www/html/www.myapp.com/vendor/symfony/console/Application.php(820): Illuminate\Console\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#12 /var/www/html/www.myapp.com/vendor/symfony/console/Application.php(187): Symfony\Component\Console\Application->doRunCommand(Object(Illuminate\Queue\Console\WorkCommand), Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#13 /var/www/html/www.myapp.com/vendor/symfony/console/Application.php(118): Symfony\Component\Console\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#14 /var/www/html/www.myapp.com/vendor/laravel/framework/src/Illuminate/Foundation/Console/Kernel.php(121): Symfony\Component\Console\Application->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#15 /var/www/html/www.myapp.com/artisan(35): Illuminate\Foundation\Console\Kernel->handle(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#16 {main}

this error keeps repeating

the jobs are stuck, nothing advances, nothing gets moved to failed-queue neither

the only solution is that I have found is stopping the supervisor workers via cronjob every couple minutes, and starting them again. Which is far from optimal

seems as well as if the whole server disappears once in a while quickly. Maybe a memory problem? i have 1 gb of memory for my live server.

I had a similar issue with beanstalkd queues. After a few hundred in it would just stop firing. I had supervisor running and nothing. The only thing that would get them working again was a server reboot. I moved over to Amazon SQS. The messages aren't in any order, but they do get fired consistently. It is pretty straight forward to get it running as well. — whoacowboy
– whoacowboy, Commented Apr 11, 2017 at 20:45
i saw my server running out of memory actually when running the jobs... — Toskan
– Toskan, Commented Apr 12, 2017 at 2:32
Post your schedule function from app\Console\kernel.php and how you run the cron job. — Dimitri Mostrey
– Dimitri Mostrey, Commented Apr 12, 2017 at 6:46

Laurence · Accepted Answer · 2020-02-29 02:42:12Z

15

It is probably one of two things:

1. Memory Leak

If you are running your queues as a daemon process, then it is possible that over time they will run out of memory. If this occurs, you have three choices.

Find a stop the memory leak
Switch to using queue:listen to ensure PHP restarts each time
Run a scheduler that includes $schedule->command('queue:restart')->hourly(). This gives you the best of both worlds, because you get the benefit of a daemon process, and just restart it once an hour to allow PHP to reset.

2. Failed database connection

There is currently a known issue that if a DB connection goes away during a daemon process, it may not reconnect. A PR is in process here - so check if this gets accepted and the bug should be fixed: https://github.com/laravel/framework/pull/19080

edited Feb 29, 2020 at 2:42

answered May 6, 2017 at 22:51

Laurence

60.3k22 gold badges178 silver badges215 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

tirta keniten Over a year ago

Thank you, this is what I'm looking for. My cPanel running outta memory so it has been suspended. I'll updated for the progress later.

RohitS Over a year ago

Any further update on this? I am facing this issue in one of my jobs.

Collectives™ on Stack Overflow

when using laravel queue job processing, getting stuck

1 Answer 1

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related