CDbException error on sending multiple campaigns

M

mailmole

Member
I've been having issues recently with my mailwizz. I migrated to a new server and allocated ca. 1tb of space for hosting my mailwizz to make sure it has enough resources.

I left it for a month or so and came back to start testing some campaigns. I started 6 different campaigns on a sample of 15k/each. However, as I started multiple campaigns at the same time, I got the CDbException error. Here's the screenshot:

46c80f7941fd42d76ad831a5c89ed32b.png


After a couple of minutes, the page went back to normal and I was able to access my mailwizz again. But only 3 of the 6 campaigns were sending something. The rest had the same status, Sending, but they weren't sending anything for hours.

I stopped all the campaigns to check and optimize my cron settings. This is what I set it to:

441f62c0861f6f06998c441fab9aa9a9.png

https://i.gyazo.com/441f62c0861f6f06998c441fab9aa9a9.png

So, with 1k of Campaigns AT Once and PCNTL enabled, I reckoned this would solve the issue.

I went back to my forntend and resumed all of my campaigns. A minute later I refreshed and got the same old CDbException error. After a few mins, I refreshed again and got my mailwizz campaigns page back on screen. But, again, only 2 of the 6 campaigns were sending any volume. I checked again an hour later and nothing changed.

This issue has been similar to a few others I have been having over the past few months. I did many things. I changed the server, updated the mailwizz version, got more space. Yet, nothing seems to be helping resolve this issue.

Has anyone has a similar issue? Where else should I look?
 
Your server simply cannot handle the settings.
1000 campaigns in parallel with 1000 subscriber batches in parallel is huge, you should not have more than 10 in both places.
Those 1000x1000 will eat all the file descriptors available on your linux box which will render other processes, like mysql, unusable.
This is why we have this: https://kb.mailwizz.com/articles/un...ery-settings-and-how-they-impact-performance/

As i said numerous times, mailwizz works in batches, so lower number will always perform better.
 
@twisted1919 Thank you for the explanation. I followed your advice yesterday and tested a new setup to no success. Here's the screenshot:

1954d7f5af838c4dfe8a6f07eb74b25f.png

The problem still remains. The system is sending only to 2 out of 6 campaigns. And, that is on a server, not a vps!

This serious puts me in suspicion that the issue is in the application itself. I hope this is not the case. Can you please help us here? What information do you need from us to give us new ideas for resolving this as this is impacting out business.
 
If i'd had a penny for each time when i heard the issue is the application ... :P
Anyway, you can try to debug the send campaigns command and see what it says: https://kb.mailwizz.com/articles/debug-send-campaigns-command/
My advise is to also pause your campaigns, remove the apps/common/runtime/mutex folder then try again. My hunch is that some of the campaigns remained lock from previous attempts when you had high settings, so removing the mutex folder should fix the problem.
 
mailmole said:
@twisted1919 Thank you for the explanation. I followed your advice yesterday and tested a new setup to no success. Here's the screenshot:

1954d7f5af838c4dfe8a6f07eb74b25f.png

The problem still remains. The system is sending only to 2 out of 6 campaigns. And, that is on a server, not a vps!

This serious puts me in suspicion that the issue is in the application itself. I hope this is not the case. Can you please help us here? What information do you need from us to give us new ideas for resolving this as this is impacting out business.
Click to expand...
Hi,

Try to keep Campaigns at once 5, Campaigns in parallel 4 and Subscriber batches 3.
And see how it works and see resources utilization on server.
 
twisted1919 said:
If i'd had a penny for each time when i heard the issue is the application ... :p
Anyway, you can try to debug the send campaigns command and see what it says: https://kb.mailwizz.com/articles/debug-send-campaigns-command/
My advise is to also pause your campaigns, remove the apps/common/runtime/mutex folder then try again. My hunch is that some of the campaigns remained lock from previous attempts when you had high settings, so removing the mutex folder should fix the problem.
Click to expand...

@twisted1919 Lol, sorry didn't mean to read as if I came to a definite conclusion. I followed your advice and debugged the send campaigns command. Here's the output:

root:~# /usr/bin/php -q /var/www/domain.com/apps/console/console.php send-campaigns --verbose=1
[2019-05-19 21:35:09] - Starting the work for this batch...
[2019-05-19 21:35:09] - Loading 10 campaigns, starting with offset 0...
[2019-05-19 21:35:09] - No campaign found, stopping.
[2019-05-19 21:35:09] - This cycle completed in 0 seconds and used 0 B of memory!
[2019-05-19 21:35:09] - CPU usage in last minute: 0.27, in last 5 minutes: 0.20, in last 15 minutes: 0.17!

I also deleted the directory mutex and re-started the campaigns. Unfortunately, none of that helped. We currently have 6 campaigns running at the same time, but only 1-2 have been sending today according to the logs.

What I noticed is that, when we start all the 6 campaigns, 5 of them will be sending within the first 15 mins. Within the next 30 mins, only 3 of them are sending. And, by the time it's been an hour since they were started, only one or two campaigns are sending. I noticed this pattern pretty consistently over the past 2 days.

Any further thoughts? Where else shall we look next to resolve this?

Vpul Shah said:
Hi,

Try to keep Campaigns at once 5, Campaigns in parallel 4 and Subscriber batches 3.
And see how it works and see resources utilization on server.
Click to expand...

Tried it, didn't work :(
 
Last edited:
mailmole said:
root:~# /usr/bin/php -q /var/www/domain.com/apps/console/console.php send-campaigns --verbose=1
[2019-05-19 21:35:09] - Starting the work for this batch...
[2019-05-19 21:35:09] - Loading 10 campaigns, starting with offset 0...
[2019-05-19 21:35:09] - No campaign found, stopping.
[2019-05-19 21:35:09] - This cycle completed in 0 seconds and used 0 B of memory!
[2019-05-19 21:35:09] - CPU usage in last minute: 0.27, in last 5 minutes: 0.20, in last 15 minutes: 0.17!
Click to expand...
Did you disable the regular cron job which sends the campaigns? Because the above simply says there's nothing to send, so you need to call that when you have campaigns ready for sending, and then you'll see lots of info not just these few lines.

try that and let's see how it goes.
 
twisted1919 said:
Did you disable the regular cron job which sends the campaigns? Because the above simply says there's nothing to send, so you need to call that when you have campaigns ready for sending, and then you'll see lots of info not just these few lines.

try that and let's see how it goes.
Click to expand...

@twisted1919 I did exactly as it was mentioned in your previous reply and in the linked article.

I first paused all the campaigns. Waited a few mins to make sure all stopped in the delivery logs. Then, ran crontab -e, removed the cron for send-campaigns and then ran the usr/bin/php -q command.
 
@mailmole - yeah but the cron job finds nothing to send, which means nothing is scheduled to go out at that point. Maybe schedule a campaign in the past so it starts sending when you manually call the send-campaigns command from command line.
 
@twisted1919 Here's what I did:

I created two new campaigns next to the 8 currently running. I proceeded with the 2 new campaigns all the way to the scheduling date and time. Unfortunately, in mailwizz, you can't just schedule. You have set the date and time and then click Send. There's no Save button. So, I clicked Send. Once I went back to my Campaigns, I immediately pressed on Pause so the campaigns never went out. I then paused the rest of the campaigns.

Then, I went to the command line and stopped the cron. After that I ran /usr/bin/php -q /var/www/mymailwizz.com/apps/console/console.php send-campaigns --verbose=1

and once again got this

[2019-05-23 21:40:20] - Starting the work for this batch...
[2019-05-23 21:40:20] - Loading 10 campaigns, starting with offset 0...
[2019-05-23 21:40:20] - No campaign found, stopping.
[2019-05-23 21:40:20] - This cycle completed in 0 seconds and used 0 B of memory!
[2019-05-23 21:40:20] - CPU usage in last minute: 0.67, in last 5 minutes: 0.33, in last 15 minutes: 0.20!

All this while following your instruction step by step. If I am missing anything please point this out because I think I did everything you tole me to do and yet no output. Should I now resume the paused campaigns? Or, is there something else I should do next?

BTW, should we really have all the campaigns stopped and cron disabled? I tried
* * * * * /usr/bin/php -q /var/www/mymailwizz.com/apps/console/console.php send-campaigns >/dev/null 2>&1
with cron disabled but campaigns triggered to send in the browser and it starting giving the output.
 
Last edited:
@mailmole - we're on the right direction. can you try to schedule the campaign in the past, like one day ago? Maybe the time on the server isn't properly set and it takes some time for those 2 campaigns to go out, therefore scheduling them in the past will make sure they go out at the first iteration.
 
@twisted1919

I created 3 new campaigns. All 3 campaigns were scheduled to go out 2 days in past. I started them at the same time. Then, I went to the mailwizz server and ran /usr/bin/php -q /var/www/mw3.safetrkpath.co/apps/console/console.php send-campaigns --verbose=1 without crontab -e on the server (otherwise doesn't work).

got the following output:

[2019-06-04 23:33:26] - Starting the work for this batch...
[2019-06-04 23:33:26] - Loading 10 campaigns, starting with offset 0...
[2019-06-04 23:33:26] - Found 3 campaigns and now starting processing them...
[2019-06-04 23:33:26] - Since PCNTL is active, we will send 20 campaigns in parallel and for each campaign, 10 batches of subscribers in parallel.
[2019-06-04 23:33:27] - Campaign Worker #1 looking into the campaign with ID: 59
[2019-06-04 23:33:27] - Campaign Worker #1 looking into the campaign with ID: 61
[2019-06-04 23:33:27] - Campaign Worker #1 looking into the campaign with ID: 62
[2019-06-04 23:33:27] - This campaign belongs to Adminuid: zz843g6sc2b4c).
[2019-06-04 23:33:27] - Changing the campaign status into PROCESSING!
[2019-06-04 23:33:27] - This campaign belongs to Admin(uid: zz843g6sc2b4c).
[2019-06-04 23:33:27] - Changing the campaign status into PROCESSING!
[2019-06-04 23:33:27] - This campaign belongs to Admin(uid: zz843g6sc2b4c).
[2019-06-04 23:33:27] - Changing the campaign status into PROCESSING!
[2019-06-04 23:33:27] - Cannot find a valid server to send the campaign email, aborting until a delivery server is available!Campaign UID: rw807mr5kbc66
[2019-06-04 23:33:27] - Cannot find a valid server to send the campaign email, aborting until a delivery server is available!Campaign UID: gn212ofz0m019
[2019-06-04 23:33:28] - Cannot find a valid server to send the campaign email, aborting until a delivery server is available!Campaign UID: nt677cwtbmea5
[2019-06-04 23:33:30] - This cycle completed in 4.21 seconds and used 2 MB of memory!
[2019-06-04 23:33:30] - CPU usage in last minute: 0.22, in last 5 minutes: 1.00, in last 15 minutes: 0.76!

The problem still remains. Mailwizz keeps sending just one of the 3 identical campaigns (the only different is a list).
 
It says your campaign don't have Delivery server to use for sending campaign.

Can you check that Delivery server is assigned or have enough hourly limit to send out campaigns.
 
@twisted1919 any other suggestions? As I said, the campaigns were set to start 2days in past, each had 3 servers running at 1k/hr. But the delivery logs show that only now campaign is running. What else could be going wrong?
 
@mailmole - i don't know what to say, the message that you got is related to the fact the app cannot find a delivery server to send those emails, so that's the issue. Now, why it can't find a delivery server is something we have to find out.
You said you ran:
Code: 
 /usr/bin/php -q /var/www/mw3.safetrkpath.co/apps/console/console.php send-campaigns --verbose=1
can you also run:
Code: 
 /usr/bin/php -v
And see if the version it outputs is the same version that you see in Backend > Misc > PHP Info?
Also, what types of delivery servers are you using? Is there anything special to them, like quotas or things like this?
 
@twisted1919 I ran the command and got the following output:

PHP 7.2.19-0ubuntu0.18.04.1 (cli) (built: Jun 4 2019 14:48:12) ( NTS )
Copyright (c) 1997-2018 The PHP Group
Zend Engine v3.2.0, Copyright (c) 1998-2018 Zend Technologies
with Zend OPcache v7.2.19-0ubuntu0.18.04.1, Copyright (c) 1999-2018, by Zend Technologies

Had a look in Backend and saw it says PHP Version 7.2.17-0ubuntu0.18.04.1.

Is this really a big difference?

Regarding the delivery servers, they are all SMTPs and they were set up in the same way as in the version before migration. I checked inside the servers and everything is working fine. Each campaign is using the same set of servers. So if one is working, the rest should too.
 
It seems like there's something wrong with crons or deep in the backend. I have just tried to import a 45k list (2mb) for the first time since the migration and it's taking hours!
 
When you migrate from old server to new - have you check that cron set properly with new server settings.

Also if your old server is running then make it off and then see.

I saw in past that after migration, cron is finding old server and it not working with migrated server. So check that settings.
 
You must log in or register to reply here.
Back
Top