Mailwizz on High Availbility Clusters of Servers (Both DB & Application)

[pradeep sharma]

Dear @twisted1919

We have implemented mailwizz behind a Load Balancer to to implement high Availability application systems
Both at database level(using MYSQL MASTER-MASTER MASTER-SLAVE database cluster and a LoadBalancer and it works perfects) & at Application Level. But Now we are facing a bit difficulty while implementing Clustering at application level

Problem :
There are some cache files/Import/export csv files/ generated on application server at public_html/apps/common/runtime/ directory which need to be synced to another application running in master-master Configuration in real time with no time lag. in our scenario each request by the user is distributed in round rubin to cluster of applications running behind load balancer. So files generated on APP-1 application are not found and retrieved by APP-2.

can we implement separate file/cache server where all the cache generated by user or application is saved/retrieved at run time so there is no need to sync cache file over each application server.
if we can implement this can you give us few tips to implement this. My Developers can implement this but we are not sure how the rest of things and feature will be behave after this.

Can we implement a separate server(keeping directory structure same) to save files of User HTML Template/Images and Template Gallery by Admin as thsi is also a road block to implement High Availability Cluster of Application.

in NutShell We need to achieve that application should not generate any file in runtime or anytime on its local server. if files are generated by application in runtime then it should be saved in remote file server to keep consistency while rotating the user request over cluster of application.

Note: Thanks for implementing & Saving Session data in DB as it was another one of the biggest roadblock to implement HA application which is already solved.
You can also implement or add this option as a feature in future release if possible. it will solve the roadblock of Mailwizz on Cluster and will increase the scalability at horizontal level.

 

[twisted1919]

Hey,

Sounds like avery interesting setup you have there and i think i can help.
If you look in the runtime directory, a big part of the issue is the /cache folder which contains various app cached data.
Good news is that this is file cache AND you can change it to use Redis cache or database cache, whatever works better for you.
See this article for info: http://www.yiiframework.com/doc/guide/1.1/en/caching.overview
Right now in apps/common/config/main.php you have:

[...]
'cache' => array(
    'class' => 'system.caching.CFileCache',
    'keyPrefix' => md5('v.' . MW_VERSION . Yii::getPathOfAlias('common')),
),
[...]

You can override this in your main-custom.php file by doing, for example, this:

[...]
'cache'=>array(
    'keyPrefix' => '123', //
    'class'=>'CRedisCache',
   'hostname'=>'localhost',
   'port'=>6379,
   'database'=>0,
   'options'=>STREAM_CLIENT_CONNECT,
),
[...]

Now, the issue remains the other files generated in the runtime directory, such as mutex folder and so on.
What you can do here, is to setup rsync and sync the generated files with the other apps. You'd also have to do the same with the assets files generated automatically in the {frontend,backend,customer}/assets/cache folders.

Can we implement a separate server(keeping directory structure same) to save files of User HTML Template/Images and Template Gallery by Admin as thsi is also a road block to implement High Availability Cluster of Application.

I do have an extension for Amazon S3 that saves the email assets to S3. I can upload it to CC if you're interested.

I guess that's all i can say

 

[pradeep sharma]

Thanks @twisted1919 for your reply..
i ll either setup memcache or Redis for my implementation but for time being to test the setup i disabled caching by using CDummyCache
i tired to add the cache details in main-custom.php to get the desired result then it was not overriding then later i added it in main.php then it was implemented and there was no cache files are generated on apps/common/runtime/cache

but there are some more cache files are generated at following directory locations
../public_html/frontend/assets/cache
../public_html/backend/assets/cache
../public_html/customer/assets/cache

how can we take care of this as well? can we either stop it or implement your suggestion in these location directory as well

Next...
if we dont want to use Amazon S3 to store customer data file like html images and wish to serve/update/upload it to a remote server?
can u suggest some guide lines or files/piece of code we need to see..

 

[twisted1919]

how can we take care of this as well? can we either stop it or implement your suggestion in these location directory as well

use rsync for these, it's the only way i can think of.

if we dont want to use Amazon S3 to store customer data file like html images and wish to serve/update/upload it to a remote server?

There is a action filter called "email_template_upload_behavior_handle_upload_before_save_content" where you can hook into when templates get uploaded, i.e:

Yii::app()->hooks->addAction('email_template_upload_behavior_handle_upload_before_save_content', function($params) {
            // for easier access
            $template        = $params['template'];
            $originalContent = $params['originalContent'];
            $storagePath     = $params['storagePath'];
            $storageDirName  = $params['storageDirName']; 
            $cdnSubdomain    = $params['cdnSubdomain'];

            if ($cdnSubdomain || sha1($template->content) != sha1($originalContent)) {
                return;
            }

           // your parse and upload logic here...
});