Campaigns stuck after server reboot

[someone]

had to reboot the server for reasons not related to MW and now the campaigns are stuck on "Sending" and "Pending Sending". MW is the latest version, the delivery servers are active, the server has plenty of CPU and RAM - Mailwizz was working ok before the reboot.

[2022-03-19 20:02:57] - Starting the work for this batch...
[2022-03-19 20:02:57] - Loading 9 campaigns, starting with offset 0...
[2022-03-19 20:02:57] - Found 7 campaigns and now starting processing them...
[2022-03-19 20:02:57] - Since PCNTL is active, we will send 5 campaigns in parallel and for each campaign, 5 batches of subscribers in parallel.
[2022-03-19 20:02:57] - Starting pre-checks for campaigns...
[2022-03-19 20:02:57] - [Worker 2] Campaign id 63 pre-checks: Starting...
[2022-03-19 20:02:57] - [Worker 1] Campaign id 59 pre-checks: Starting...
[2022-03-19 20:02:57] - [Worker 3] Campaign id 64 pre-checks: Starting...
[2022-03-19 20:02:57] - [Worker 1] Campaign id 59 pre-checks: Populating customer data...
[2022-03-19 20:02:57] - [Worker 2] Campaign id 63 pre-checks: Populating customer data...
[2022-03-19 20:02:57] - [Worker 4] Campaign id 65 pre-checks: Starting...
[2022-03-19 20:02:57] - [Worker 5] Campaign id 71 pre-checks: Starting...
[2022-03-19 20:02:57] - [Worker 3] Campaign id 64 pre-checks: Populating customer data...
[2022-03-19 20:02:57] - [Worker 4] Campaign id 65 pre-checks: Populating customer data...
[2022-03-19 20:02:58] - [Worker 5] Campaign id 71 pre-checks: Populating customer data...
[2022-03-19 20:04:58] - [Worker 3] [WARNING] Campaign id 64 pre-checks: Unable to acquire mutex on line 819!
[2022-03-19 20:04:58] - [Worker 4] [WARNING] Campaign id 65 pre-checks: Unable to acquire mutex on line 819!
[2022-03-19 20:04:58] - [Worker 2] [WARNING] Campaign id 63 pre-checks: Unable to acquire mutex on line 819!
[2022-03-19 20:04:59] - [Worker 5] [WARNING] Campaign id 71 pre-checks: Unable to acquire mutex on line 819!

I emptied the mutex folder couple of times, but I always get stuck here.

So, I re-enabled the cron job and left it as is. Today (around ~24 later) the campaigns are still stuck. Even those that were scheduled for today (~24h after the reboot)

[2022-03-20 18:44:13] - Starting the work for this batch...
[2022-03-20 18:44:13] - Loading 9 campaigns, starting with offset 0...
[2022-03-20 18:44:13] - Found 8 campaigns and now starting processing them...
[2022-03-20 18:44:13] - Since PCNTL is active, we will send 5 campaigns in parallel and for each campaign, 5 batches of subscribers in parallel.
[2022-03-20 18:44:13] - Starting pre-checks for campaigns...
[2022-03-20 18:44:13] - [Worker 1] Campaign id 59 pre-checks: Starting...
[2022-03-20 18:44:13] - [Worker 2] Campaign id 63 pre-checks: Starting...
[2022-03-20 18:44:13] - [Worker 3] Campaign id 64 pre-checks: Starting...
[2022-03-20 18:44:13] - [Worker 4] Campaign id 65 pre-checks: Starting...
[2022-03-20 18:44:13] - [Worker 5] Campaign id 71 pre-checks: Starting...
[2022-03-20 18:44:13] - [Worker 2] Campaign id 63 pre-checks: Populating customer data...
[2022-03-20 18:44:13] - [Worker 4] Campaign id 65 pre-checks: Populating customer data...
[2022-03-20 18:44:13] - [Worker 3] Campaign id 64 pre-checks: Populating customer data...
[2022-03-20 18:44:13] - [Worker 1] Campaign id 59 pre-checks: Populating customer data...
[2022-03-20 18:44:13] - [Worker 5] Campaign id 71 pre-checks: Populating customer data...
[2022-03-20 18:44:15] - [Worker 2] Campaign id 63 pre-checks: Pre-counting campaign subscribers...
[2022-03-20 18:44:15] - [Worker 3] Campaign id 64 pre-checks: Pre-counting campaign subscribers...
[2022-03-20 18:44:15] - [Worker 3] Campaign id 64 pre-checks: Pre-counting campaign subscribers, start counting...
[2022-03-20 18:44:15] - [Worker 5] Campaign id 71 pre-checks: Pre-counting campaign subscribers...
[2022-03-20 18:44:15] - [Worker 5] Campaign id 71 pre-checks: Pre-counting campaign subscribers, start counting...
[2022-03-20 18:44:15] - [Worker 4] Campaign id 65 pre-checks: Pre-counting campaign subscribers...
[2022-03-20 18:44:15] - [Worker 4] Campaign id 65 pre-checks: Pre-counting campaign subscribers, start counting...
[2022-03-20 18:44:15] - [Worker 1] Campaign id 59 pre-checks: Pre-counting campaign subscribers...
[2022-03-20 18:44:15] - [Worker 1] Campaign id 59 pre-checks: Pre-counting campaign subscribers, start counting...
[2022-03-20 18:44:15] - [Worker 2] Campaign id 63 pre-checks: Pre-counting campaign subscribers, start counting...
[2022-03-20 18:44:19] - [Worker 4] Campaign id 65 pre-checks: Pre-counting campaign subscribers, finished counting in 4.43057 seconds...
[2022-03-20 18:44:19] - [Worker 4] Campaign id 65 pre-checks: Finding campaign max subscribers count and adjusting quota left...
[2022-03-20 18:44:19] - [Worker 4] Campaign id 65 pre-checks: Currently quota left is 9223372036854775807 and campaign available subscribers for this batch is 0...
[2022-03-20 18:44:19] - [Worker 4] Campaign id 65 pre-checks: Done!
[2022-03-20 18:44:19] - [Worker 3] Campaign id 64 pre-checks: Pre-counting campaign subscribers, finished counting in 4.49110 seconds...
[2022-03-20 18:44:19] - [Worker 3] Campaign id 64 pre-checks: Finding campaign max subscribers count and adjusting quota left...
[2022-03-20 18:44:19] - [Worker 3] Campaign id 64 pre-checks: Currently quota left is 9223372036854775807 and campaign available subscribers for this batch is 0...
[2022-03-20 18:44:19] - [Worker 3] Campaign id 64 pre-checks: Done!
[2022-03-20 18:44:19] - [Worker 2] Campaign id 63 pre-checks: Pre-counting campaign subscribers, finished counting in 4.55066 seconds...
[2022-03-20 18:44:19] - [Worker 2] Campaign id 63 pre-checks: Finding campaign max subscribers count and adjusting quota left...
[2022-03-20 18:44:19] - [Worker 2] Campaign id 63 pre-checks: Currently quota left is 9223372036854775807 and campaign available subscribers for this batch is 0...
[2022-03-20 18:44:19] - [Worker 2] Campaign id 63 pre-checks: Done!

Now this...

Any idea?

 

[twisted1919]

Please see your ticket where we recommended enabling debug mode to see what is going on.