Debug transactional process?

[Kane]

How do we debug the "send-transactional-emails" command? Adding --verbose=1 just gives an errror.

Our transactional email queue sometimes is backed up for 20 min, and when I manually run the process there's still a bunch of items stuck at "Unsent". I can't figure out what's going on. Also the cron is set to run every minute.

/backend/settings/cron => "Send at once" = 100, and there's not 100 waiting in the queue either.

Any ideas?

 

[twisted1919]

The only way to debug that command is to simply edit the file(apps/console/commands/SendTransactionalEmails.php) and echo debug messages there and then to run the command and see the debug messages.

 

[Kane]

Can you think of any reason why, when manually running the command via SSH, there's still a bunch of Unsent items? And it completes in like 1 second...almost like it does nothing.

 

[twisted1919]

i do have a hunch,
In the process() method you have:

protected function process()
{
    $offset = (int)Yii::app()->options->get('system.cron.transactional_emails.offset', 0);
    $limit = 100;

try to make it:

protected function process()
{
    Yii::app()->options->set('system.cron.transactional_emails.offset', 0); // this line added.
    $offset = (int)Yii::app()->options->get('system.cron.transactional_emails.offset', 0); 
    $limit = 100;

Then save the file and run the command from command line only once and see if it starts processing.
If it does, open the file and comment out the line you have added.
if it doesn't, we'll look into other possible issues.

 

[twisted1919]

Actually you know what, replace the entire process() method with this one:

protected function process()
    {
        $emails = TransactionalEmail::model()->findAll(array(
            'condition' => '`status` = "unsent" AND `send_at` < NOW() AND `retries` < `max_retries`',
            'order'     => 'email_id ASC',
            'limit'     => 100,
        ));
       
        if (empty($emails)) {
            return $this;
        }

        foreach ($emails as $email) {
            $email->send();
        }

        Yii::app()->getDb()->createCommand('UPDATE {{transactional_email}} SET `status` = "sent" WHERE `status` = "unsent" AND send_at < NOW() AND retries >= max_retries')->execute();
        Yii::app()->getDb()->createCommand('DELETE FROM {{transactional_email}} WHERE `status` = "unsent" AND send_at < NOW() AND date_added < DATE_SUB(NOW(), INTERVAL 1 MONTH)')->execute();
       
        return $this;
    }

 

[Kane]

Ok we've determined that it's because the process is running into blacklisted emails.

The problem is though...say it's set to send 100 emails. If all 100 of those are blacklisted, it actually will send 0 emails. This seems like a bug...it should keep processing the queue until it SUCCESSFULLY sends 100, not stop when it only evaluates 100.

Now our transactional queue is backed up to about 45 min. It's constantly getting worse. I'm really surprised nobody else has had this problem so far, our list is only 70k.

So what we have now is new users that sign up and don't get their account activation email for 45 min...that's scary! Any ideas how to fix it?

 

[Kane]

FYI, we solved this by adding a line to the TransactionalEmail model to delete the TransactionalEmail record if it's found to be a blacklisted email address.

if (EmailBlacklist::isBlacklisted($this->to_email)) {
                $this->delete();
            return false;
}

Do you think this is ok? Should it be added to the core code?

 

[twisted1919]

Hey,

Great catch actually.
How about, beside your codefix, we add the code before saving the model, this way we don't have to store it in the database just to delete it later.
Here's the change i have done in the model file for the beforeSave() method:

protected function beforeSave()
    {
        if (empty($this->plain_text) && !empty($this->body)) {
            $this->plain_text = CampaignHelper::htmlToText($this->body);
        }
        if (empty($this->email_uid)) {
            $this->email_uid = $this->generateUid();
        }
        // below block added.
        if (EmailBlacklist::isBlacklisted($this->to_email)) {
            $this->addError('to_email', Yii::t('transactional_emails', 'This email address is blacklisted!'));
            return false;
        }
        return parent::beforeSave();
    }

and of course the send() method now contains:

        [...]
        $server = $servers[$cid];
       
        if (!$server->canSendToDomainOf($this->to_email)) {
            return false;
        }
       
        if (EmailBlacklist::isBlacklisted($this->to_email)) {
            $this->delete(); // your line.
            return false;
        }
        [...]