Parallel sending explained - The correct configuration

F

Fradian

New Member
Hello everyone,

I have a problem when running different campaigns in the same time and I spent lots of time on the forum but couldn't find a reliable solution.

I have a VPS server with 4gb RAM. I'm using sparkpost to send my emails using 4 dedicated IPs. In Mailwizz I have setup 4 campaigns that should run in parallel but for some strange reason the sending pattern is extremely unpredictable, a server sends way more than the others then switch randomly to another server where it sends just a few emails and so on. Something like this:
Campaign 1...............23%
Campaign 2...............5%
Campaign 3...............6%
Campaign 4...............35%
They all start at the same hour, and instead of sending uniformly it focus only on two of them sending emails like mad, and the rest of two being left behind with just a few sends, forcing me to pause the campaigns and resume when the others catch up.

My main problem is that the emails are being delayed on ISP's and I can get into trouble. I'm using Mailwizz just for myself, I don't have clients and stuff.

Q1: What is the priority between these 3 setup pages in backend?
Customers--->Groups
Settings ---> Customers ---> Campaigns
Settings ---> Cron
I'm asking this because on each of these pages you can set subscribers at once, send at once....etc, but when the cronjob run, which of the above pages has priority and apply the settings? I don't have groups, the only customer is me.

Q2: I'm still in the warm-up process and after 1 week I reached 4000 emails/day/ip but it is a real struggle not to send this amount in a too short period of time. I aim to spread this number to 4-5 hours.
My current setup in Settings ---> Customers ---> Campaigns is:
Subscribers at once: 10
Send at once: 20
Pause: 0
Emails per minute: 0
Change server at: 20
My current setup in Settings ---> Cron
Campaigns at once: 100
Subscribers at once: 5
Send at once: 0
Pause: 0
Emails per minute: 0 (This option would help me the most but looks like it doesn't work when "change server at" has a value in it)
Campaigns in parallel: 10
Subscriber batches in parallel: 4 (in the idea of 4 batches * 5 subscribers at once = 20)
Parallel sending via PCNTL is off. Turning it on doesn't send campaigns in parallel but one after another
The cron I have setup is this one:
*/6 * * * * ..../apps/console/commands/shell/send-parallel-campaigns > /dev/null 2>&1

The question is: How do I setup the above so that the campaigns are sent in parallel, uniformly, and in a predictable pattern?

Thank you
 
Q1 :
Customers--->Groups => 1st prio
Settings ---> Customers ---> Campaigns =>2nd prio
Settings ---> Cron => 3rd prio ... but not used because 2nd prio

Q2 :

Use send command with --verbose=1 to see how it work
 
Thanks for your answer.

I didn't have setup any group, I created now one and this means the cronjob will apply the settings from here and just from here? I mean, it will not go further down to Settings ---> Customers ---> Campaigns or Settings ---> Cron to fetch the settings from there right?
 
yes.

For your Q2, as I can see the shell launch X (max=16) send-campaigns command.
I think with pcntl = off, Campaigns in parallel: 10 and Subscriber batches in parallel: 4 are not applied.
 
Matt95c said:
yes.

For your Q2, as I can see the shell launch X (max=16) send-campaigns command.
I think with pcntl = off, Campaigns in parallel: 10 and Subscriber batches in parallel: 4 are not applied.
Click to expand...

So in this case the cronjob will apply the settings from Settings-->Cron only if pcntl is on? But if I set the pnctl to ON, the campaigns stop sending in parallel but one after another.
 
Because you use shell send-parallel-campaigns who exec send-campaigns with params --campaigns_limit=1

Try with classic cron command send-campaigns

(make some test with verbose mode to find best setting)
 
Ok, switched back to classic cron and activated PNCTL with Campaigns in parallel:10 and Subscriber batches in parallel:4
Tha cron job is set to */3.

But for me, to better understand, in this situation what is the priority order?

So on each 3 minutes the cron will apply the above settings from CRON page or it will apply the ones from Customer --> Groups?

On Customer-->Group I have:

Subscribers at once: 5
Send at once: 5
Pause: 0
Change server at: 0

What does the cron do? It will come at every 3 minute, take 5 subscribers and send the message to them, then wait another 3 minutes when the cron will return and take another 5 subscribers and so on?

Thanks

PS: I'm looking in -verbose 1 but still cant figure out which one does what
 
"So on each 3 minutes the cron will apply the above settings from CRON page or it will apply the ones from Customer --> Groups?"
Setting from customer groups. (ask to @twisted1919 for utility of setting of subscriber at once on cron page, for me that's not usable due to customer->campaign)

"What does the cron do?
PNCTL with Campaigns in parallel:10 and Subscriber batches in parallel:4
Tha cron job is set to */3.
On Customer-->Group I have:

Subscribers at once: 5
Send at once: 5
Pause: 0
Change server at: 0"

Each 3 minutes, if not send-campaign in use, search 10 campaigns to send and for each campaign send 4 batches of 5 subscribers.
if you've got only 1 campaign active : 1 campaign * 4 batches * 5 subscribers = send 20 emails
if you've got only 4 campaigns active : 4 campaigns * 4 batches * 5 subscribers = send 80 emails
if you've got 10 campaigns active : 10 campaigns * 4 batches * 5subscribers = send 200 emails

PS : Setting depends cpu, ram and delivery server, but 5 subs seems ridiculously small
 
Yes 5 subs it's really small but I set it this way so that I won't get to much delays.

Btw, from your experience, would I get into trouble with the ISP's (yahoo for example) if they get to much emails from me at once ( ex. 2000 in one our)? I'm still in warm-up process so I'm not sure what will happen if they constantly have to delay my messages.

Thanks again for your help
 
I don't know about sparkpost, I use my own delivery server.
They don't set limit by isp ? (so you need to make it via mailwizz ?)
 
Fradian said:
would I get into trouble with the ISP's (yahoo for example)
Click to expand...

Thats true, big ISP count that.

but I would not worry about it, because I'm using port25 pmta.

So I use parallel sending to feed emails to pmta as fast as possible. then from pmta I'm configuring it to send only certain number (Message speed rate(per hour/min/day), simulate connection rates, # of msg per connection, recipient per message). this will give you control on your sending. I believe other MTA has almost the same control over sending.
 
frm.mwz said:
One would guess they do have all of the necessary policies/directives, as their mother company is port25 / messagesystems (the pmta and momentum producers), but it does not hurt to ask them if the respect the receiving limits.
If you read carefully, it is covered here:
https://www.sparkpost.com/features/email-deliverability/
Click to expand...

I've sent them yesterday an email asking how they handle the emails received from mailwizz, if they send all the emails right away or they put them in a queue releasing them gradually based on the ISP's limits. Unfortunately their support is dead slow and I didn't got an answer.

But, reading on their blog I found an article here where they explain the Accepted metric from their dashboard.
"Targeted: The number of messages you sent SparkPost.

Sent: The number of messages that have been delivered (or attempted to deliver) by SparkPost (includes multiple delivery attempts, including all In-Band Bounces).

Accepted: The number of messages that were accepted by the receiving server."

Now on my statistics I see that mailwizz sent 6228 emails to sparkpost. These emails appear as "Targeted". Next there is the Sent metric which shows only 3882 emails sent ( ~60%) from the Targeted amount.

This raise my big question, does Sparkpost intentionally delay some of the emails based on inner calculations and ISP's limits or they send the whole amount to the ISP which delays the surplus messages based on it's limit policy.

In this case, how healthy will be in time for my sending reputation to constantly get delays?

Sorry guys for bugging you with my sparkpost issue but I'm a beginner into this and probably they wont answer real soon to my ticket. And if they say that the system can receive all the emails from Mailwizz and then send them gradually based on ISP limits..................this will solve my problem for which I opened this thread in the first place and I think it will not be necessary to buy pmta which i guess it is pretty expensive.
 

Attachments

  • sparkpost.jpg
    sparkpost.jpg
    128.2 KB · Views: 30
This raise my big question, does Sparkpost intentionally delay some of the emails based on inner calculations and ISP's limits or they send the whole amount to the ISP which delays the surplus messages based on it's limit policy.
Click to expand...

That's Sparkpost job.
Limit sending to 4K/day is a nonsense in warmup process, if you send 4K/day only to gmail or 4K/day on 10 ISPs that's don't have same effect.(limit must be by isp and global)

Sparkpost know how it work and I think they warmup your ips with their experience.
 
I was sending that amount per dom
Matt95c said:
That's Sparkpost job.
Limit sending to 4K/day is a nonsense in warmup process, if you send 4K/day only to gmail or 4K/day on 10 ISPs that's don't have same effect.(limit must be by isp and global)

Sparkpost know how it work and I think they warmup your ips with their experience.
Click to expand...

Now I realize indeed it was nonsense to limit the number of emails/day/domain in the warm up process, especially with Sparkpost which turns out to be an incredible service. Basically all my messages from above are nonsense, now that I learned how the entire system work.

Looks like some times being way to cautious is not that good :)
 
Very interesting thread for newbees like me. Is it possible to send the same campaign through more than one email service? With all the limitations in terms of emails per minute/hour/day it would be greaat to send one campaign using more than one delivery server.
Hope there's a way to do this.
 
jorgepelaez said:
Very interesting thread for newbees like me. Is it possible to send the same campaign through more than one email service? With all the limitations in terms of emails per minute/hour/day it would be greaat to send one campaign using more than one delivery server.
Hope there's a way to do this.
Click to expand...
that is really not a question re "parallel sending", but, yes, you can do that, but that spreading across services will actually make it more likely to be filtered out...as it is oft interpreted as to be similar to hiding/snowshoewing...
 
You must log in or register to reply here.
Back
Top