Process campaigns using Redis Queue.

@Rodrigo Bustos - I don't know how that could happen, that's been fairly tested at large scale without issues. Any change the redis connection was not properly setup, or maybe using same redis database as other apps ?
Did you tried it with less batches?
 
twisted1919 said:
@Rodrigo Bustos - I don't know how that could happen, that's been fairly tested at large scale without issues. Any change the redis connection was not properly setup, or maybe using same redis database as other apps ?
Did you tried it with less batches?
Click to expand...

Mailwizz runs in a dedicated server with Redis inside. Just normal OS/Mailwizz updates there are no changes in config, etc.

Some months ago we found some duplicates in our external smtp but never found the guilty.. our first thought was our MySQL Cluster (Galera, wsrep or anything related to deadlocks) but we couldn't find anything related.

This time we deactivated Redis just for testing purposes due it reach 100% cpu usage when Mailwizz began to process a campaign and perform batches and it kept like this meanwhile the campaign was running (quite odd). So we tried with different numbers and combinations with same behavior. Finally without Redis it ran perfectly. As last test we leave redis as cache but mutexes and all went fine. So something odd happens when mutex is used to generate batches under unknown delivery setups yet. (number of batches, emails per batch, etc)

Below an example (1 single campaign running for testing):

$cat user@domain 2019-04-24.log
[2019-04-24 13:02:16] - user@domain - 407/500
[2019-04-24 13:02:16] - Checking if we can send to domain of user@domain...
[2019-04-24 13:02:16] - Checking if user@domain is still confirmed...
[2019-04-24 13:02:16] - Checking if user@domain is blacklisted...
[2019-04-24 13:02:16] - Checking if user@domain is listed in a campaign suppression list...
[2019-04-24 13:02:17] - Done for user@domain, logging delivery...
[2019-04-24 13:03:11] - user@domain - 118/500
[2019-04-24 13:03:11] - Checking if we can send to domain of user@domain...
[2019-04-24 13:03:11] - Checking if user@domain is still confirmed...
[2019-04-24 13:03:11] - Checking if user@domain is blacklisted...
[2019-04-24 13:03:11] - Checking if user@domain is listed in a campaign suppression list...
[2019-04-24 13:03:13] - Done for user@domain, logging delivery...
[2019-04-24 13:06:39] - user@domain - 500/500
[2019-04-24 13:06:39] - Checking if we can send to domain of user@domain...
[2019-04-24 13:06:39] - Checking if user@domain is still confirmed...
[2019-04-24 13:06:39] - Checking if user@domain is blacklisted...
[2019-04-24 13:06:39] - Checking if user@domain is listed in a campaign suppression list...
[2019-04-24 13:06:39] - Done for user@domain, logging delivery...

Hope it helps to clarify.
 
You must log in or register to reply here.
Back
Top