YASP - Yet Another Speed Problem (Mailgun)

[SpyPower]

Hello there. While using the platform, i noticed a big problem for me with speed delivery.
i was sending 80.000 emails using a delivery server with the Mailgun Api. The campaign did 14 hours to complete, that means 1.58email/sec. The server is hosted in azure, so network connection is not an issue. a) i will post my settings, b) i want a clarification about two statuses, c) i will post some settings and the results afterwards.


a) Current settings of the clients group:

Subs at once 2k, emails per minute 2k, everything else disabled.
Here is the cron settings (which runs every minute as i assume)​

4 campaigns at once (rarely more than 2 at the same time, subs at once 80, send at once 80, pause 10sec, emails per minute 300. PCNTL will not and cannot be used on this current server.​

From what i understand, this client with the above client group, will be throttled down from the cron job, regardless of what settings I insert on his group, since emails per minute allowed by cron is 300, (and this will not be taken into consideration = 80 x 6 (10sec interval) = 480email/min). Is my assumption right? This needs to be answered.
Another major question i have to ask is, under settings there is a customer tab also. If a client is within a customer group, will these settings be applied? AND, in which order are the settings applied? 1st Customer group settings/Client settings, 2nd Cron ? 1st customer group settings, 2nd client settings, 3rd cron?

b)While i was sending the 80k subs campaign, at the start the status was pending-sending, then it changed to sending, and after 2 minutes or so, it changed to processing. From the status i assume that the sending was throttling down. is that right? I had to pause and restart the campaign to change the status to sending, but again after 2 minutes again processing. When it was at processing status, it was sending with a rate of 180email/minute at maximum.

c) The next day, confused AF, i did some testing. I removed my account from any group ( i assume here that the limitation of the group does not apply to my account, but the limitation under settings->customer->domain applies) and here are the settings:

Subs at once 5k, everything else disabled.
cron job as before, 4 campaigns at once (rarely more than 2 at the same time, subs at once 80, send at once 80, pause 10sec, emails per minute 300. PCNTL will not and cannot be used on this current server.​

Delivery server Mailgun.
With these settings i was able to send 138email/minute, like the previous campaign sent problem i encountered.
BUT!, when i changed the customer settings to

5 subs at once, pause 10sec,
Then i was able to send emails to 5 subs but not once every 10 or 30/min, but 5 subs EVERY MINUTE.

I am confused AF. I contacted mailgun, and they told me that can have 4500 api requests per minute and they can go up to 9k/minute.
I did some other tests with another provider (sendgrid) and 1k emails were sent in under 1minute with those exact settings.
The client is registered and verified on the domain so we must exclude this problem. I am using version 1.3.6.5 currently.

ps. When i was sending the 80k campaign, i did a
/usr/bin/php -q /var/www/mail.apifon/apps/console/console.php send-campaigns --verbose=1
and i had big waiting timings between the sending the email and getting the OK response. This waiting timings varied from 3 seconds to 30 seconds.
Here is a log.

[2016-10-27 13:20:10] - *********@gmail.com - 29/10000
[2016-10-27 13:20:10] - Checking if we can send to domain of *********@gmail.com...
[2016-10-27 13:20:10] - Checking if *********@gmail.com is blacklisted...
[2016-10-27 13:20:10] - Checking if the server is over quota...
[2016-10-27 13:20:10] - Checking if the customer is over quota...
[2016-10-27 13:20:10] - Preparing the entire email...
[2016-10-27 13:20:10] - Using delivery server: ********.gr (ID: 54).
[2016-10-27 13:20:10] - Sending the email message using the DIRECT method.
[2016-10-27 13:20:14] - Sending response is: OK
[2016-10-27 13:20:14] - Sending OK.
[2016-10-27 13:20:14] - Done for *********@gmail.com, logging delivery...

Chaning to 15 subs every 10 second, it sent 15email per minute.
The cron is scheduled every minute to run, but the pause means that it has to wait a full cron cycle to restart? What's the meaning of pause then?

 

[Vpul Shah]

Hello,

For speed, I suggest to use PCNTL - ON.

 

[SpyPower]

@Vpul Shah Thanks for the effort .
<insert potato>

 

[SpyPower]

I just want to ask something specific. If you could answer me, I would be grateful.
Customer campaign = 5subs/once, 0 send/once, 0 pause, 0 emails/min.
Cron settings = 0 send/once, 1 pause, 0 mail/minute.
Should that setting result into 5 subscribers every second resulting 5*60=300subs/minute?
Mine is sending 5 subs every minute with those.

 

[Vpul Shah]

If you can specify what exactly you want to send per minute, then easy to help you.

 

[SpyPower]

2k up to 10k/min
But upon testing, i came across the problem using those settings from my previous post.

 

[frm.mwz]

2k up to 10k/min
But upon testing, i came across the problem using those settings from my previous post.

from the screen shots you can see that 'subs at once' is all over the place inconsistent, fix that first to a good number (100) to start with for testing purposes
and
read all texts that pop up when your mouse cursor is over the respective field

 

[SpyPower]

Thank you for your replies.
On my screenshots above I deliberately posted pictures with inconsistent numbers in order to get feedback of what exactly these are doing.
Even reading the text above, while i mouse over the setting, does not answer lots of questions. Ex.

• Subscribers at once (How many subscribers to process at once for each loaded campaign. per cron? Are subs loaded for process? are to be send?
  
• How many emails should we send before pausing 0 for disabled. (if 0, then how many will be send at once? If only 1 campaign is loaded with unique subs on each campaign, then this setting will be taken into consideration or the subs at once? If yes, then how the pause and the cron works?
  
• Pause, how many seconds to sleep after a batch of emails. (where does it say how big is the batch? is it the subs at once or the send at once? what happens with the cron pause? is this pause inside the interval of the cron job?)