How to automatically process Redis queue

Lakjin

Active Member
MailWizz can send emails very quickly using it's Redis queue feature. (If you don't know what Redis queue is or don't have it enabled, go read https://forum.mailwizz.com/threads/process-campaigns-using-redis-queue.216/ first.) However, currently, the way to process Redis queue is a bit cumbersome and error prone if the Redis queue processes are killed; or, if they aren't killed, they just continue running even if they don't have anything to process. So I came up with a quick script to help solve this issue. See below.

------------------

NOTE: I'm a noob, so please correct any errors or mistakes in the script. Also feel free to suggest any way to improve the script. Thanks!

Throw the following code into a PHP script:

Code:
<?php

exec("/usr/bin/redis-cli scan 0 | /bin/grep 'queue:emails-queue'", $out);
$t = count($out);

exec("/bin/ps aux | /bin/grep 'resque-1.2'", $out2);
if ($t >= 1) {
   if (count($out2) <= 3) {
     exec("/usr/bin/nohup /usr/bin/php-cli -q /xxx/apps/console/console.php queue --workers=15 --interval=5 --verbose=0 >/dev/null 2>&1 &");
   }
} else {
   if (count($out2) > 2) {
     exec("/usr/bin/pkill -f 'resque-1.2'");
   }
}

die();
?>

Note that in the script you may need to modify the paths to each for the commands, e.g. ps, grep, redis-cli, etc. according to your own server settings. You can do the following to get the absolute paths:

Code:
whereis redis-cli
whereis grep
whereis ps
whereis nohup

Also note if you have MailWizz to use a Redis database other than 0, you will need to modify the above script accordingly.

After that, simply create a cron and run it every ten minutes (or at whatever interval you want):

Code:
*/10 * * * * /usr/bin/php-cli -q /path/to/script.php >/dev/null

Now every ten minutes the script will run; it will look and see if the proper key value is found in redis to indicate something needs to be processed. If it is found, then it will check to see if redis queue processes are already running and start them if they aren't. If it is not found, then all redis queue processes are killed.
 
Last edited:
  • Like
Reactions: Rob
Thanks for this, I'll give it a whirl and see what happens.
Great! I tried it myself; there must be some bug in the code because when running it manually, it works. However, when running it from cron, it won't start new processes. The rest of the code works fine in cron.
 
Okay, I figured out why the script wasn't working -- need absolute paths for cron. Updated now and it works just fine.
 
Back
Top