Problem with auto-completion of custom fields

Groundarker

Groundarker

Member
Dear Mailwizz staff,
I come back to you to ask you some explanations about the synchronization function of the custom fields.

After the request that I have expressed to you here: https://forum.mailwizz.com/threads/auto-completion-of-custom-fields-created-for-a-list.6632/
I kindly asked you to insert TAGs to auto-fill in the last email opening date, last click opening date, last sent date, etc.

I thank you for the implementation and it seems that these TAGs work.

The problem I objectively encounter now is this:
I have 2 lists of which list n.1 is sent by setting the function of moving the user towards list n.2 in the sending schedule.

The user is correctly moved but the field last opened date, last click date, etc. remain the initial ones and the system does not update this data!

I tried to search the forum if someone else, like me, had the same problem as me but I found only one suggestion here: https://forum.mailwizz.com/threads/creating-segments-from-field-with-no-value.4206/

Here you say you can use the following cron command every 30 minutes
Code: 
php -q apps/console/console.php sync-lists-custom-fields --verbose=1
and this command after running it has practically created values with apostrophes in fields where nothing appeared,

I attach screenshots below:

1590663689896.png



However even running this command the fields have not been updated! Could you please verify this problem?
Thanks as always and good job!

See you soon
 
twisted1919 said:
@Groundarker - could you please open a support ticket with backend url and access to your app so we can run some tests on it?
Click to expand...
Guys, I confirm that I was able to solve the problem on my own. Practically launching the daily cron manually does not create that mess as I sent you in the screenshot, while launching the cron that I indicated in the opening thread it messes up all the data.

The only problem remains the huge amount of data that the cron must process being millions of contacts and having at least about 10 custom fields or more takes a very long time, is there any way to speed up the process eventually?
Thanks a lot!
 
@Groundarker - I think we have also fixed this on our end. Also processing the custom fields will work much faster with the upcoming release.
So let us know how things work after 1.9.10 update.
 
Guys @twisted1919 @laurentiu , thank you for releasing the latest version of the software.
I'll be back here to inform you that I have had the opportunity to install and try it but by launching the custom field sync the waiting times are really long to autocompile the data of millions of subscribers.
Can you by chance do something to make this function more performing?

I ask you this because we (mainly) use the following fields filled in by default: SUBSCRIBER_LAST_OPEN_DATETIME, SUBSCRIBER_LAST_CLICK_DATETIME, SUBSCRIBER_LAST_SEND_DATETIME.

As you can imagine, we then segment this field in the list to be able to retrieve who has opened or clicked any newsletter in the last 30/60/90/120 days. If updating the custom fields takes a long time in updating the above fields, the segment will consequently lose many contacts that could actually fall within the range of days.

You have certainly improved your performance with the latest update, but with millions of contacts under management the process takes a long time to process! I really hope you will find another alternative solution for this problem ...

As always good work and congratulations for this fantastic project!
 
@Groundarker - Thank you for looking into this, i can't tell you how much this feedback is appreciated ;)
While we're super busy with other tasks, i'll make sure we will improve this one for the next version, i think we can implement parallel processing via pcntl like we do for lots of heavy commands ;)
I will keep you posted.
 
@Groundarker - following up with this, here's the speed from your version, that is for 100k subscribers and updating a custom field:
Code: 
[2020-06-22 10:12:52] - Batch is done!
[2020-06-22 10:12:52] - Loading subscribers set for the list with limit: 1000 and offset 100000
[2020-06-22 10:12:52] - Done, no more subscribers for this list!
[2020-06-22 10:12:52] - Done!
docker/php/runcmd.sh sync-lists-custom-fields --verbose=1  
0.14s user 0.10s system 0% cpu 
14:26.00 total

And this is the improvement:
Code: 
[2020-06-22 09:56:33] - Batch is done!
[2020-06-22 09:56:34] - Loading subscribers set for the list with limit: 1000 and offset 100000
[2020-06-22 09:56:34] - Done, no more subscribers for this list!
[2020-06-22 09:56:34] - Done!
docker/php/runcmd.sh sync-lists-custom-fields --verbose=1  
0.09s user 0.09s system 0% cpu 
53.330 total

So we're down from 14 minutes to under 1 minute.
We're including this in the next release.
 
Good afternoon guys and well found!
I hope all right in your part :)

I am writing to you because I have just updated and tried the new version with some obvious improvements regarding the issues raised! You are fantastic, I hope so much that you can speed it up even more than that. To process my database it took 1 day and a half, considering I processed the process on 3/4 kk of data with about 40 custom fields.

I highlight problems regarding the automatic compilation of the last fields you added, including:

  • [SUBSCRIBER_LAST_OPEN_DATETIME]
  • [SUBSCRIBER_LAST_CLICK_DATETIME]
  • [SUBSCRIBER_LAST_SEND_DATETIME]

I proceeded to insert it in the default value and activate the auto-update option of the default custom fields, but the problem persists.
Not all data is processed correctly.

Could you please verify this kindly?
Thank you
 
Is there any chance we get access to the app and also ssh so we can test exactly what you're doing? That would help more than having us go blindly through changes. If that's possible, maybe open a support ticket with all the info we need to reproduce this on your end.
 
You must log in or register to reply here.
Back
Top