Slow sending... Again - max counts ignored

henkedk

henkedk

Member
Hey guys & @twisted1919

So, another thread in regards to slow sendouts...

I've been testing a lot - first things first - HW:
4 core CPU(8 threads)
32 GB ram
SSD
100 Mbit uplink
Jessie, PHP5, Nginx 1.8.1, MySQL 5.5 <- yes, I know, old soft, but I have other stuff running on the server that still depend on old version.
External PMTA SMTP

MySQL performance is solid and optimized, even so the queries and structure of how MW plans campaigns/selects the recipients could be heavily optimized, this is not the bottleneck.

I ofc followed the KB for understanding sendout settings and their effect on the system, did some different testing with/without pcntl - at the very last, I disabled the cron and set 500 subscribers at once in the settings, disabled pcntl and ran the send-campaigns --verbose=1 manually.

This is where I was a bit surprised - MW only selects 100 subscribers and sends to those - not the 500 I have pushed in the settings. I checked the settings DB, it says 500 alright - reran the command a couple of times - same result, always just 100 selected and sent.

So - this ofc makes me wonder if anyone has an idea to what's going on with my install?

Each of the sent mail takes a little less than a second - which would be acceptable if the pcntl setup would be working solid - but it isn't - at least not for me. Sending 10k mails easily takes 45 minutes.

Other than when the cron runs once a minute and MW hits the MySQL heavy for some seconds, the server is steady at a low CPU usage, so it should be able to put through a lot more than it currently does.

Any ideas on what steps to take to optimize the speed?
 
@henkedk - a quick one, mailwizz checks the settings as follows.

If the customer is in a group, it looks in the group settings for the number of subscribers at once.
If the customer is in no group maybe you have set the details in backend > settings > customers area.
Last place where mailwizz looks, is in backend > settings > cron.

So have a look, somewhere you have 100 subscribers at once.
 
Hi @twisted1919
Nr 2 had the setting, bumped that to 500 while still having it on 200 in the cron settings. When running the verbose, it now selects 500, so why is the setting for subscribers at once in the cron settings, when it is ignored anyway?

Running verbose, it sends arpox 2 mails a second, so 100 subscribers should probably be the sweetspot, as it would complete before running the next cron.

But, back to the real problem - reenabling pcntl and cron, I still get very slow sending speeds - had the max subscribers at 300 (which seems to be ignored), 5 campaigns in parallel with 5 subscriber batches - I'm pushing through ~80-150 mails a minute.

Any suggestions?
 
@henkedk - try more subscriber batches in parallel, that should open more smtp connections in parallel and delivery more.
You can also relay your emails to a local email server, then let that server delivery them in it's own pace.
 
@twisted1919
Settings currently:
Campaigns at once: 50
Subscribers at once: 200 in cron settings, 500 in usr settings
5 Campaigns in parallel, 5 subscriber batches in parallel

Sending at ~8k mails an hour... Waaaay to slow :/
 
henkedk said:
@twisted1919
Settings currently:
Campaigns at once: 50
Subscribers at once: 200 in cron settings, 500 in usr settings
5 Campaigns in parallel, 5 subscriber batches in parallel

Sending at ~8k mails an hour... Waaaay to slow :/
Click to expand...
What SMTP relay or ESP are you using? They may be a limiting factor. Some SMTP relays / ESPs have high latency and will decrease the speed at which MailWizz sends emails.
 
Hmmm campaigns sent to segments seem to be the problem, bc campaigns sent to the entire list blast out at 50+ mails a second...
 
I took a look at your SQL structure etc @twisted1919 - the way we built the sender in my former company was to make one big select of the profiles for the campaign that is set to start, put all profile IDs in a job/tmp table and base your minute send on that instead of reselecting the entire subscriber table every time - should take a lot of load of the server and give you the possibility to send out fast no matter what kind of job it is.
 
henkedk said:
I took a look at your SQL structure etc @twisted1919 - the way we built the sender in my former company was to make one big select of the profiles for the campaign that is set to start, put all profile IDs in a job/tmp table and base your minute send on that instead of reselecting the entire subscriber table every time - should take a lot of load of the server and give you the possibility to send out fast no matter what kind of job it is.
Click to expand...
What about the load for the initial query, if it has to scan few K records and put them in another table ?
I can imagine something like "SELECT subscriber_id FROM list_subscriber WHERE xyz...zzzz" will be damn slow if the list has, say 500k records...
 
twisted1919 said:
What about the load for the initial query, if it has to scan few K records and put them in another table ?
I can imagine something like "SELECT subscriber_id FROM list_subscriber WHERE xyz...zzzz" will be damn slow if the list has, say 500k records...
Click to expand...
Well yeah that select would be rather slow, but it only runs once for each campaign, and ofc you have to put in as many limits as possible - SQL is actually pretty fast if it doesn't have to give you data back and you run as stored procedures - makes more sense than punishing the SQL every minute :)
 
henkedk said:
Well yeah that select would be rather slow, but it only runs once for each campaign, and ofc you have to put in as many limits as possible - SQL is actually pretty fast if it doesn't have to give you data back and you run as stored procedures - makes more sense than punishing the SQL every minute :)
Click to expand...
With optimized settings you can still do a few thousand per minute for a heavily segmented mail out.
 
We set a local pickup server, would it still be much slower if a client sets to sent for a segment of his ~100k emails?
Also, you think it is more taxing for the DB to have many different segments, or to have 1 segment with many conditions?
 
Niko said:
We set a local pickup server, would it still be much slower if a client sets to sent for a segment of his ~100k emails?
Click to expand...
100k is not that much so it should be fine.
Niko said:
Also, you think it is more taxing for the DB to have many different segments, or to have 1 segment with many conditions?
Click to expand...
Smaller segments will perform better than one big segment.
 
You must log in or register to reply here.
Back
Top