112,000 emails in my bounce inbox - and rising

S

Simon Brown

New Member
Hi,

I have 112,000 messages in my bounce inbox.

Yesterday (when it was only 45,000 messages) I went into Emergency Actions and pressed to change from cron_running to active. It's only made it worse (unsurprisingly).

Is there a script I can run from the command line to deal with this backlog? To process all the bounces, kick them out of the queue, get them off my mail server. And then how do I get bounce proccessing back to working properly?

(server specification, 8GB RAM, 4 Cores, 80GB SSD, only running Mailwizz nothing else)

Thanks

Simon
 
Hi Simon,

As it stays right now we don't have any other command line tool to process the bounces.
Is the smtp server on the same server with mailwizz?
You use only a single bounce server?
 
Hi Twisted,

No that box is dedicated to MailWizz. (8GB RAM, 4 cores)
The website runs on a different box - again it only runs the website. (2GB RAM, 1 core)
The SMTP server runs on a 3rd box, we use the SMTP server (it runs iredmail) for bounces as well. (4GB RAM, 2 cores)
It has dovecot and postfix on it, as per a standard iredmail install.
I have a 2nd SMTP server ready to go. (6GB RAM, 6 cores).

The script that's called in cron - can it be run directly from the command line to see if it can help? Or at least return some sort of error?

We're now up to ~130,000 emails in there...
 
I see, you should be able to call the bounce-handler from the cron, just have a look at Backend -> Misc -> Cron jobs list, take it from there and paste it in the console. Without the frequency(*/10 * * * *) and the end part that says >/dev/null...
In your iredmail server do you see mailwizz connecting to get the bounces?
 
I ran it without the -q as well to try to get some output - it runs for 10 seconds then exits.
I also tried > ~/out.txt but it makes an empty file.

ah: on the iredmail server:
grep bounce /var/log/dovecot.log gives lots of Oct 15 15:08:53 imap(bounce@artisan-mail.com): Error: mmap() failed with file /var/vmail/vmail1/artisan-mail.com/b/o/u/bounce-2015.10.02.15.29.46//Maildir/dovecot.index.cache: Cannot allocate memory

That won't be helping...
 
vsz_limit wasn't set, default is 64Mb. On a 3GB mailbox that's not going to play nice. I've upped it temporarily to 1024MB which we handle the backlog. Hopefully that should do it.

At least manually running the bounce-handler from the MailWizz server doesn't seem to stop after a few seconds.
 
Simon Brown said:
vsz_limit wasn't set, default is 64Mb. On a 3GB mailbox that's not going to play nice. I've upped it temporarily to 1024MB which we handle the backlog. Hopefully that should do it.

At least manually running the bounce-handler from the MailWizz server doesn't seem to stop after a few seconds.
Click to expand...

Still won't allocate memory, going to temporarily upgrade the RAM on the mail server to handle the backlog
 
temporarily on 16GB RAM and 8 cores. Still won't load the cache file. Deleted the cachefile and restarted Dovecot. I try running the script from the Mailwizz box and it stops after 10 seconds. How can I diagnose this?
 
Simon Brown said:
temporarily on 16GB RAM and 8 cores. Still won't load the cache file. Deleted the cachefile and restarted Dovecot. I try running the script from the Mailwizz box and it stops after 10 seconds. How can I diagnose this?
Click to expand...
155,000 emails now, mailbox over 3GB. Can you help?
 
The mailwizz script will create a lock file so that not many instances of it can run at the same time (if it would, it would screw things)
So if the scripts ends after 10 sec, means some other instance of it already runs.
You can run ps aux | grep php to see what's running php related.
Not sure how i could help if the issue is dovecot related?
 
It was dovecot related - the cache file was too large. In fact I had to pause all campaigns for 3 days. The bounces would process a little bit at a time - maybe 50 every 10 minutes. After 3 days the bounce list had reduced down to manageable proportions.

I am moving the bounces over to Gmail, using a forwarded address. In the meantime I'm alarmed that my VPS whose only job is running Mailwizz is so slow with mail server access. Spec is 8GB RAM, 40GB SSD, 4 cores, nothing else running on this server except Mailwizz. Not very impressive...
 
OK - thanks for the tip, I'll definitely do that. The MySQL database gets very big very quickly ;)
Any other tips? Or shall I do that first and then come back ;)
 
Yes indeed the database gets very big very quickly because we store so much data.
One way is to simply rmeove campaigns that you don't need anymore, that will really really make your server fast again as it will remove huge amounts of data that you don't use anymore anyway.
keep in mind that actual removal of campaigns is done by the cron job that runs at midnight. When you delete a campaign from the web interface, what it does is to simply remove it from view, but it will be kept in database till midnight.

P.S: Whatever it is, run mysqltuner on your database and follow its advices.
 
Still having massive problems with bounces. 300,000 of them now.

Code: 
According to the manual:

Bounce servers are optional Yes, this is true.
You are not forced to add and use a bounce server, but if you don't, then the entire bouncing mechansim will not work, that is, you will not get statistics about campaign bounces and you will never know which email address is good and which not.
Please note that you can use a GMAIL account or any other provider as your bouncing account.

Problem is, that means the email will be marked as "From: blah-random-account@gmail.com" - not "From: no-reply@spf-and-dkim-domain.com"

I've set a "forwarder" up so that "no-reply@spf-and-dkim-domain.com" points to blah-random-account@gmail.com.

Then when I set up my bounce server... how do I set the bounce address? is it "From" address?
 
Simon Brown said:
Still having massive problems with bounces. 300,000 of them now.

Code: 
According to the manual:

Bounce servers are optional Yes, this is true.
You are not forced to add and use a bounce server, but if you don't, then the entire bouncing mechansim will not work, that is, you will not get statistics about campaign bounces and you will never know which email address is good and which not.
Please note that you can use a GMAIL account or any other provider as your bouncing account.

Problem is, that means the email will be marked as "From: blah-random-account@gmail.com" - not "From: no-reply@spf-and-dkim-domain.com"

I've set a "forwarder" up so that "no-reply@spf-and-dkim-domain.com" points to blah-random-account@gmail.com.

Then when I set up my bounce server... how do I set the bounce address? is it "From" address?
Click to expand...

Add a bounce address under bounce servers first. Then, under delivery servers, you'll be able to select the bounce address/server you created under bounce servers. Your From: address will be whatever you input in the "From email *" under delivery servers (default) or what you input in the actual campaign. Your bounce address will be listed in "Return-Path:"

Note: If this IP/domain you're sending from is new, you can't send that large a list in the beginning. You'd have no reputation. Failure to properly warm the IP would have you in throttle/deferred hell to where it starts damaging your IP reputation.
 
I set up a Gmail email as a bounce server and it's not working. No bounces are being recorded and they should be - I'm only getting partial data here.
Eg: Open rate is 0% but click rate is 0.165%,
Eg: zero bounces but 12 delivery errors.

Any and all advice gratefully received here... I can open the Gmail box remotely in Thunderbird but not sure if it's not opening on Mailwizz, or what the problem could be...
 
You must log in or register to reply here.
Back
Top