Making PHP's mail() asynchronous

Question

I have PHP's mail() using ssmtp which doesn't have a queue/spool, and is synchronous with AWS SES.

I heard I could use SwiftMail to provide a spool, but I couldn't work out a simple recipe to use it like I do currently with mail().

I want the least amount of code to provide asynchronous mail. I don't care if the email fails to send, but it would be nice to have a log.

Any simple tips or tricks? Short of running a full blown mail server? I was thinking a sendmail wrapper might be the answer but I couldn't work out nohup.

Telemechanics

exec('php mailcript.php ... – Quantic

idea: store the mail somewhere as queue instead of sending, and use a background script to send from the queue. – Hannon

use Threads, they are async: docs.php.net/manual/en/class.thread.php#114752 – Consolatory

Could you please provide a code example of using threads to quickly send mail() off in the background please? – Telemechanics

Does the normal sending with PHPs mail() function works fine for you? – Throttle

I want the least amount of code to provide - SO is not a place to beg for code. This "question" should already be closed. Using "bounty" to prevent as you did simply sucks. – Lanthanum

If you want to queue mails and send them asynchronously, just switch to a smtpd that has one. Why reinvent the wheel? – Boser

mail() works but it's synchronous @JRsz. I don't want to run a full smtpd. I'm using ssmtp. – Telemechanics

Since you are using Amazon SES, why dont you just use the API? It allows for async sending. Cf. docs.aws.amazon.com/aws-sdk-php/v3/api/… – Finned

@Telemechanics I have used nohup and generated separate script for async functionality. Do you want to use nohup then I can put code as an answer. – Sardonic

Parixit, please contribute the nohup wrapper / sample. I'm interested. – Telemechanics

Idea : Its does not matter which php mailer you are using. You just have to set "TO" (Mail id to whom you want to send email) with comma and send mail. Try it. – Ellga

Lol, I tried it. It took 5.1seconds. – Telemechanics

POST the email message to SNS instead. Have SNS trigger the actual email sending code upon new message. – Hbeam

I don't want to depend on AWS. – Telemechanics

Answer 1

php-fpm

You must run php-fpm for fastcgi_finish_request to be available.

echo "I get output instantly";
fastcgi_finish_request(); // Close and flush the connection.
sleep(10); // For illustrative purposes. Delete me.
mail("[email protected]", "lol", "Hi");

It's pretty easy queuing up any arbitrary code to processed after finishing the request to the user:

$post_processing = [];
/* your code */
$email = "[email protected]";
$subject = "lol";
$message = "Hi";

$post_processing[] = function() use ($email, $subject, $message) {
  mail($email, $subject, $message);
};

echo "Stuff is going to happen.";

/* end */

fastcgi_finish_request();

foreach($post_processing as $function) {
  $function();
}

Hipster background worker

Instantly time-out a curl and let the new request deal with it. I was doing this on shared hosts before it was cool. (it's never cool)

if(!empty($_POST)) {
  sleep(10);
  mail($_POST['email'], $_POST['subject'], $_POST['message']);
  exit(); // Stop so we don't self DDOS.
}

$ch = curl_init("http://" . $_SERVER['HTTP_HOST'] . $_SERVER['REQUEST_URI']);

curl_setopt($ch, CURLOPT_TIMEOUT, 1);
curl_setopt($ch, CURLOPT_NOSIGNAL, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, [
  'email' => '[email protected]',
  'subject' => 'foo',
  'message' => 'bar'
]);

curl_exec($ch);
curl_close($ch);

echo "Expect an email in 10 seconds.";

Peninsula

I am using php-fpm, but that approach makes me have to structure my mail() carefully to be on the south side of fastcgi_finish_request(); I don't quite follow the curl example. You timeout the curl request even though it probably got the request out ... ?!? – Telemechanics

@Telemechanics 1. Just make an execution queue for later. I've added an example. 2. requestA send a POST to itself starting requestB. The timeout causes the curl to return almost immediately so that the user initiated request can continue executing. Now concurrently requestA continues to the echo statement and exits and concurrently requestB continues to send the email. – Peninsula

Answer 2

You have a lot of ways to do this, but handling thread is not necessarily the right choice.

• register_shutdown_function: the shutdown function is called after the response is sent. It's not really asynchronous, but at least it won't slow down your request. Regarding the implementation, see the example.
• Swift pool: using symfony, you can easily use the spool.
• Queue: register the mails to be sent in a queue system (could be done with RabbitMQ, MySQL, redis or anything), then run a cron that consume the queue. Could be done with something as simple as a MySQL table with fields like from, to, message, sent (boolean set to true when you have sent the email).

Example with register_shutdown_function

<?php
class MailSpool
{
  public static $mails = [];

  public static function addMail($subject, $to, $message)
  {
    self::$mails[] = [ 'subject' => $subject, 'to' => $to, 'message' => $message ];
  }

  public static function send() 
  {
    foreach(self::$mails as $mail) {
      mail($mail['to'], $mail['subject'], $mail['message']);
    }
  }
}

//In your script you can call anywhere
MailSpool::addMail('Hello', '[email protected]', 'Hello from the spool');


register_shutdown_function('MailSpool::send');

exit(); // You need to call this to send the response immediately

Johnnyjohnnycake

And that code executes quickly? Isn't it synchronous on shutdown? – Telemechanics

The register_shutdown_function is synchronous, but it is executed AFTER the response is sent to the client, so it does not matter if the sending takes 10s for instance. – Johnnyjohnnycake

Takes almost 5 seconds in my test for the response to complete. :( – Telemechanics

Your test might be wrong. Maybe you can pastebin your whole test – Johnnyjohnnycake

exit doesn't exit the process and close the connection when a shutdown function is registered. If you replaced the last 2 lines of code with MailSpool::send(); the exact same thing would be happening. – Peninsula

Had to sent an email after status change, did with register_shutdown_function and it suited my needs. Thank you. – Introvert

Thank you for your answer. I ended up using this method for sending emails asynchronously. However, I am having one big problem. Some emails are being CC'd to mails that don't have to be there and that causes confusion and frustration. Why's that? – Oppidan

register_shutdown_function was't executing after the response was sent for me (php 7), so I used this solution – Contraceptive

Answer 3

Use AWS SES with PHPMailer.

This way is very fast (hundreds of messages per second), and there isn't much code required.

$mail = new PHPMailer;
$mail->isSMTP();                                      // Set mailer to use SMTP
$mail->Host = 'ssl://email-smtp.us-west-2.amazonaws.com';  // Specify main and backup SMTP servers

$mail->SMTPAuth = true;                               // Enable SMTP authentication

$mail->Username = 'blah';                 // SMTP username
$mail->Password = 'blahblah';                           // SMTP password


$mail->SMTPSecure = 'tls';                            // Enable TLS encryption, `ssl` also accepted
$mail->Port = 443; 

Not sure if i interpreted your question correctly but i hope this helps.

Selectman

Still makes the Web response takes four seconds. :/ – Telemechanics

It might not help for this case but also a great solution I think. Thanks. – Immesh

It won't send emails asynchronously. – Tui

Answer 4

Pthreads is your friend :)
This is a sample of how i made in my production application

class AsynchMail extends Thread{
    private $_mail_from;
    private $_mail_to;
    private $_subject;

    public function __construct($subject, $mail_to, ...) {
        $this->_subject = $subject;
        $this->_mail_to = $mail_to;
        // ... 
    }
    // ...
    // you must redefine run() method, and to execute it we must call start() method
    public function run() {
        // here put your mail() function
        mail($this->_mail_to, ...);
    }
}

TEST SCRIPT EXAMPLE

$mail_to_list = array('[email protected]', '[email protected]',...);
foreach($mail_to_list as $mail_to) {
    $asynchMail = new AsynchMail($mail_to);
    $asynchMail->start();
}

Let me know if you need further help for installing and using thread in PHP
For logging system, i strongly advice you to use Log4PHP : powerful and easy to use and to configure
For sending mails, i also strongly advice you to use PHPMailer

Mordancy

Be good to get a self contained sample in a file so I can run and test it please! – Telemechanics

Why would you use threads if you already have php-fpm available? You just chose to occupy a cpu core by spawning a thread from a process, instead of letting that process do its synchronous work. Threads do absolutely nothing useful here in terms of.. anything except wasting resources. – Allover

@Allover According to the php documentation: Warning The pthreads extension cannot be used in a web server environment. Threading in PHP should therefore remain to CLI-based applications only. – Trevatrevah

@Trevatrevah so why are you higlighting me exactly? I don't really follow. The extension was made so it can be used with mod_php, or php-fpm, it still can if you know what to change at compile time. – Allover

Answer 5

I'm using asynchronous php execution by using beanstalkd.
It is a simple message queue, really lightweight and easy to integrate.

Using the following php wrapper for php https://github.com/pda/pheanstalk you can do something as follows to implement a email worker:

use Beanstalk\Client;
$msg="dest_email##email_subject##from_email##email_body";

$beanstalk = new Client(); 
$beanstalk->connect();
$beanstalk->useTube('flux'); // Begin to use tube `'flux'`.
$beanstalk->put(
    23,  // Give the job a priority of 23.
    0,   // Do not wait to put job into the ready queue.
    60,  // Give the job 1 minute to run.
    $msg // job body
);
$beanstalk->disconnect();

Then the job would be done in a code placed into a separate php file.
Something like:

use Beanstalk\Client;
$do=true;

try {
    $beanstalk = new Client();
    $beanstalk->connect();
    $beanstalk->watch('flux');

} catch (Exception $e ) {
    echo $e->getMessage();
    echo $e->getTraceAsString();
    $do = false;
}

while ($do) {
    $job = $beanstalk->reserve(); // Block until job is available.
    $emailParts = explode("##", $job['body'] );

    // Use your SendMail function here

    if ($i_am_ok) {
        $beanstalk->delete($job['id']);
    } else {
        $beanstalk->bury($job['id'], 20);
    }
}
$beanstalk->disconnect();

You can run separately this php file, as an independent php process. Let's say you save it as sender.php, it would be run in Unix as:

php /path/to/sender/sender.php & && disown

This command would run the file and alsow allow you to close the console or logout current user without stopping the process.
Make sure also that your web server uses the same php.ini file as your php command line interpreter. (Might be solved using a link to you favorite php.ini)

I hope it helps.

Seaquake

8k LOC of C for beanstalkd and a package for something I expect to take one line of code? No. – Telemechanics

I just depends if you want a quick whatever o a integral solution. I answer for the second one, just in case. – Seaquake

You're better of using nohup rather than disown unix.stackexchange.com/a/148698/3125 – Peninsula

good to read, very useful. For this example it works with & && disown because the executed process does not get input from the console session. – Seaquake

Answer 6

An easy way to do it is to call the code which handles your mails asynchronously.

For example if you have a file called email.php with the following code:

// Example array with e-mailaddresses
$emailaddresses = ['[email protected]', '[email protected]', '[email protected]'];

// Call your mail function
mailer::sendMail($emailaddresses);

You can then call this asynchronously in a normal request like

exec('nice -n 20 php email.php > /dev/null & echo $!');

And the request will finish without waiting for email.php to finish sending the e-mails. Logging could be added as well in the file that does the e-mails.

Variables can be passed into the exec between the called filename and > /dev/null like

exec('nice -n 20 php email.php '.$var1.' '.$var2.' > /dev/null & echo $!');

Make sure these variables are safe with escapeshellarg(). In the called file these variables can be used with $argv

Edgaredgard

How do you feed the variables of the mail() command into the exec? – Telemechanics

I have updated the post with an example of this. It may be easier to have all your needed variables in a seperate config file and just include that into the called file. – Edgaredgard

This is 1 user input away from being an RCE vulnerability. – Peninsula

Which is why you either escape user-input, or (better yet) simply do not use any user-input. – Edgaredgard

Answer 7

Your best bet is with a stacking or spooling pattern. It's fairly simple and can be described in 2 steps.

• Store your emails in a table with a sent flag on your current thread.
• Use cron or ajax to repeatedly call a mail processing php file that will get the top 10 or 20 unsent emails from your database, flag them as sent and actually send them via your favourite mailing method.

Lauren

So now I need a database ? Urgh – Telemechanics

for the best solution, yes. – Lauren

Answer 8

Welcome to async PHP https://github.com/shuchkin/react-smtp-client

$loop = \React\EventLoop\Factory::create();

$smtp = new \Shuchkin\ReactSMTP\Client( $loop, 'tls://smtp.google.com:465', '[email protected]','password' );

$smtp->send('[email protected]', '[email protected]', 'Test ReactPHP mailer', 'Hello, Sergey!')->then(
    function() {
        echo 'Message sent via Google SMTP'.PHP_EOL;
    },
    function ( \Exception $ex ) {
        echo 'SMTP error '.$ex->getCode().' '.$ex->getMessage().PHP_EOL;
    }
);

$loop->run();

Regimentals

Answer 9

Use register_shutdown_function and fastcgi_finish_request together!

There are answers using one or the other here, but you can combine them. This can be useful, for example, if you are using a CMS like WordPress or Joomla where you may not have convenient access to manage application code at the end of a request where you can place the fastcgi_finish_request() call.

According to the register_shutdown_function() documentation, you can still write to output, which means the request is still open and the client will continue waiting, so on its own, this function does not solve the problem of performing longer running tasks while allowing the client to continue on it's journey.

However, you can call fastcgi_finish_request() as the first action within a shutdown function, which then releases the client from purgatory. A simple adaptation of @magnetik's example gives us:

<?php
class MailSpool
{
  private static $mails = [];
  private static $enqueued = false;

  public static function addMail($subject, $to, $message)
  {
    // Register the shutdown function on the first message queueing request
    if (!self::$enqueued) {
      register_shutdown_function('MailSpool::send');
      self::$enqueued = true;
    } 
    self::$mails[] = [ 'subject' => $subject, 'to' => $to, 'message' => $message ];
  }

  public static function send() 
  {
    fastcgi_finish_request();
    foreach(self::$mails as $mail) {
      mail($mail['to'], $mail['subject'], $mail['message']);
    }
  }
}

// Call this from anywhere in your code
MailSpool::addMail('Hello', '[email protected]', 'Hello from the spool');

Coruscation