Extremely slow sending

L

Lakjin

Active Member
Hello,

I have about 500,000 emails to send per day, across about 5 campaigns at various different times in the day. In backend settings, I have set it to process 5000 emails in batch, send 1000 emails in batch, and no wait between. However, I ran the send command with verbose it looks like MailWizz is only sending roughly 100 emails every 40 seconds per campaign (majority of which is is time taken for "connection to remote ends"). I'm not sure, but it looks like MailWizz is processing 100 emails at a time and sending them one at a time, for some reason.

I tried http://www.mailwizz.com/article/increase-delivery-speed-recursive-or-parallel-processing with -c 50 -s 0 -t regular but it only runs two parallel processes and does nothing to speed up the process.

What is going on? Any way to speed this up?

Thanks!
 
Last edited:
Rob said:
Take a look at this post regarding the remote SMTP connection time https://forum.mailwizz.com/threads/increase-email-sending-speed.269/#post-1222

In the backend settings I would add a pause between batches.
Click to expand...
I had a pause between batches before I set it to zero 'cause I thought it would speed it up. I don't know why but it feels like the backend settings just aren't being applied to the frontend -- when I run it with verbose, the process batch is still just 100 emails even though I've set it to much higher.
 
Rob said:
Check the cron settings backend/settings/cron
Click to expand...
That is the settings I am talking about. In there, I have it set to: 5000 per processing, 1000 sending, 1 sec pause. When I run it in verbose, it shows 100 per processing.
 
There are 3 places those settings can be made, the one already mentioned above.

Settings > Customers > "Campaigns" tab
Customers > Groups >select a group and look at "Campaigns" tab
 
Rob said:
There are 3 places those settings can be made, the one already mentioned above.

Settings > Customers > "Campaigns" tab
Customers > Groups >select a group and look at "Campaigns" tab
Click to expand...
Thank you, I didn't know they were there in two places. I'll try changing this and see if it works.
 
Lakjin said:
Thank you, I didn't know they were there in two places. I'll try changing this and see if it works.
Click to expand...
Changing the setting seems to have worked -- now when I run in verbose, it is showing 5000 queue. However, email sending is still slow and it looks like emails are still being sent one at a time instead of in 1000 batch. It is taking roughly .3 secs per email, which is still very slow.
 
Lakjin said:
Changing the setting seems to have worked -- now when I run in verbose, it is showing 5000 queue. However, email sending is still slow and it looks like emails are still being sent one at a time instead of in 1000 batch. It is taking roughly .3 secs per email, which is still very slow.
Click to expand...
That's the time taken from mailwizz to your smtp server, how would mailwizz speed that up, how can mailwizz make your network transfer faster?
You basically hit an issue where no matter what you do, mailwizz defaults can't help you because that's the time it takes for making the remote connection and transfering the data from mailwizz to your smtp server.
What you need is a way so that mailwizz doesn't waste any time by waiting for remote ends, something like a queue, thus you can try the redis queue implementation for this or you can try mailwizz directory pickup, which is something more complex.

Thanks.
 
twisted1919 said:
That's the time taken from mailwizz to your smtp server, how would mailwizz speed that up, how can mailwizz make your network transfer faster?
You basically hit an issue where no matter what you do, mailwizz defaults can't help you because that's the time it takes for making the remote connection and transfering the data from mailwizz to your smtp server.
What you need is a way so that mailwizz doesn't waste any time by waiting for remote ends, something like a queue, thus you can try the redis queue implementation for this or you can try mailwizz directory pickup, which is something more complex.

Thanks.
Click to expand...
Using Redis queue fixed the issue for me.
 
Yeah, i have figured out after a while, i am answering emails/threads as they come into my email box, so i found later that your issue has been solved.
 
I run this command :
/usr/bin/php-cli -q /the/path/to/apps/console/console.php send-campaigns --verbose=1

and the result is:
2015-05-19 10:15:34] Current progress: 85 out of 1000
[2015-05-19 10:15:34] Checking if the delivery server is allowed to send to the subscriber email address domain...OK, took 0 seconds.
[2015-05-19 10:15:34] Checking the subscriber email address into the blacklist...OK, took 0.001 seconds.
[2015-05-19 10:15:34] Checking server sending quota...OK, took 0 seconds.
[2015-05-19 10:15:34] Checking customer sending quota...OK, took 0 seconds.
[2015-05-19 10:15:34] Preparing email...done, took 0.005 seconds.
[2015-05-19 10:15:34] -> Sending the email for ********@hotmail.com by using direct method...done, took 1.285 seconds.
[2015-05-19 10:15:36] The email has been sent successfully!
[2015-05-19 10:15:36] Logging delivery...done, took 0.023 seconds.

[2015-05-19 10:15:36] Current progress: 86 out of 1000
[2015-05-19 10:15:36] Checking if the delivery server is allowed to send to the subscriber email address domain...OK, took 0 seconds.
[2015-05-19 10:15:36] Checking the subscriber email address into the blacklist...OK, took 0.001 seconds.
[2015-05-19 10:15:36] Checking server sending quota...OK, took 0 seconds.
[2015-05-19 10:15:36] Checking customer sending quota...OK, took 0 seconds.
[2015-05-19 10:15:36] Preparing email...done, took 0.005 seconds.
[2015-05-19 10:15:36] -> Sending the email for ********@noavar.com by using direct method...done, took 1.141 seconds.
[2015-05-19 10:15:37] The email has been sent successfully!
[2015-05-19 10:15:37] Logging delivery...done, took 0.018 seconds.

[2015-05-19 10:15:37] Current progress: 87 out of 1000
[2015-05-19 10:15:37] Checking if the delivery server is allowed to send to the subscriber email address domain...OK, took 0 seconds.
[2015-05-19 10:15:37] Checking the subscriber email address into the blacklist...OK, took 0.001 seconds.
[2015-05-19 10:15:37] Checking server sending quota...OK, took 0 seconds.
[2015-05-19 10:15:37] Checking customer sending quota...OK, took 0 seconds.
[2015-05-19 10:15:37] Preparing email...done, took 0.005 seconds.
[2015-05-19 10:15:37] -> Sending the email for ********@gmail.com by using direct method...done, took 1.189 seconds.
[2015-05-19 10:15:38] The email has been sent successfully!
[2015-05-19 10:15:38] Logging delivery...done, took 0.018 seconds.

[2015-05-19 10:15:38] Current progress: 88 out of 1000
[2015-05-19 10:15:38] Checking if the delivery server is allowed to send to the subscriber email address domain...OK, took 0 seconds.
[2015-05-19 10:15:38] Checking the subscriber email address into the blacklist...OK, took 0.001 seconds.
[2015-05-19 10:15:38] Checking server sending quota...OK, took 0 seconds.
[2015-05-19 10:15:38] Checking customer sending quota...OK, took 0 seconds.
[2015-05-19 10:15:38] Preparing email...done, took 0.007 seconds.
[2015-05-19 10:15:38] -> Sending the email for ********@yahoo.com by using direct method...done, took 1.335 seconds.
[2015-05-19 10:15:39] The email has been sent successfully!
[2015-05-19 10:15:39] Logging delivery...done, took 0.023 seconds

and status of procces is:
ps aux | grep send
root 26530 0.0 0.1 10800 1552 pts/3 S 14:29 0:00 /bin/bash /var/www/*********t/apps/console/commands/shell/send-parallel-campaigns -s 5 -c 15 -t -p /usr/bin/php
root 26561 3.3 3.9 236500 40640 pts/3 S 14:29 0:18 /usr/bin/php -q /var/www/*********/apps/console/console.php send-campaigns --campaigns_offset=0 --campaigns_limit=1 --campaigns_type=

what is your idea about this result?
 
[2015-05-19 10:15:38] -> Sending the email for ********@yahoo.com by using direct method...done, took 1.335 seconds.
Click to expand...
If like the above, each email takes 1.3 second from your server to the smtp server, then this is your problem, the communication between servers is too slow.
 
Thanks for your answer.
I'll tell this case to my smtp service provider.
what is normal communication time for sending 1000 mail/sec?
Can you send me a real log with normal communication time?
I have problem in running parallel process still and just one process for sending campaign.
 
One way to really boost throughput speed is to:

1) Use a Solid State Drive for your server and mail spool. You will get lightning fast speeds.

2) If you cant afford that. Put your exim spool on a Ramdisk, which has the same throughput as the SSD drive but is cheaper. There is the downside that your mail queue is lost any time you reboot your server. But if you only use it for your mailings (not for other people's emails) it isn't so important. I reboot my server maybe once every other month. There are tutorials how to set this up, search in google.

I can send around 90,000 messages per hour with the second setup. I think I used to be faster back when I used the first setup, but its been a while and I can't recall the exact throughput back then.

Usually the bottleneck is in the reading and writing of thousands of tiny files So SSD is the best solution to increase speed.
 
You must log in or register to reply here.
Back
Top