How do I setup and use Laravel Scheduling on AWS Elastic Beanstalk?

Question

Scenario

As a fairly new user of Laravel and Elastic Beanstalk I soon found my self in the need to schedule operations, like most of us do.

In the past I had always used simple crontab scheduling for this. So now I stood before a list of questions:

• How do I run Laravel code using crontab?
• How do I setup crontab in my Elastic Beanstalk environment?

Finding the individual answers to these questions weren't that hard. Combining them and actually getting it all to work however turned out to be a bit tricky, which is why I've decided to share the solution here for others struggling with getting this to work properly.

---

Environment

• Laravel 5.6
• PHP 7.1

Celin

Answer 1

TL;DR:

See working .ebextentions configuration at the end of answer.

---

Environment

• Laravel 5.6
• PHP 7.1

---

How to I run Laravel code using crontab?

The answers to this question is of course the most obvious and if you're even the slightest in to Laravel you surely know the answer: Scheduling!

I won't bore you with explaining the brilliant thing that is Laravel Scheduling since you can read about it in the documentation yourself.

But the key thing we need to take with us is that Laravel Scheduling uses crontab to execute, as described in the documentation:

* * * * * php /path-to-your-project/artisan schedule:run >> /dev/null 2>&1

Which brings us to the next, and a bit more tricky, question...

---

How do I setup crontab in my Elastic Beanstalk environment?

At first glance the answer to this question may seem pretty straight forward. I found this in the AWS Knownledge Center: How do I create a cron job on EC2 instances in an Elastic Beanstalk environment?

Here they describe how to setup a cron job on your Elastic Beanstalk EC2 machine using .ebextentions. In short what it does is creating a new file in the directory /etc/cron.d/ in which we put our desired cron job.

Files in this directory is then processed by crontab as the root user. There are some of the traps I walked in to, as I've commented below:

files:

    # The name of the file should not contain any dot (.) or dash (-), this can
    # cause the script not to run. Underscore (_) is OK.
    "/etc/cron.d/mycron":

        # This permissions is important so that root user can run the script.
        mode: "000644"

        # As the file is run by the root user it needs to be the owner of the file.
        owner: root

        # For consistency it's a good idea to have root as the group aswell.
        group: root

        # NOTE: We need to explicitly tell the cron job to be run as the root user!
        content: |
            * * * * * root /usr/local/bin/myscript.sh 

# There need to be a new line after the actual cron job in the file.

Once we have stayed clear all of those traps, it's time to put in our Laravel Scheduling cron job from above. That should look something like this:

files:
    "/etc/cron.d/schedule_run":
        mode: "000644"
        owner: root
        group: root
        content: |
            * * * * * root php /path-to-your-project/artisan schedule:run >> /dev/null 2>&1

This won't really work in most cases though. That's because the Laravel Scheduler won't have access to your ENV variables and must noticeably not your database settings.

I found the answer to this here: How to Get Laravel Task Scheduling Working on AWS Elastic Beanstalk Cron

So a big shout out to George Bönnisch; I salute you sir for sharing this!

So with this last piece of the puzzle I was finally able to get the setup to work properly:

---

Working Solution

File structure:

[Project root]
    |-- .ebextensions
    |        |-- cronjob.config

cronjob.config:

files:
    "/etc/cron.d/schedule_run":
        mode: "000644"
        owner: root
        group: root
        content: |
            * * * * * root . /opt/elasticbeanstalk/support/envvars && /usr/bin/php /var/www/html/artisan schedule:run 1>> /dev/null 2>&1

commands:
    remove_old_cron:
        command: "rm -f /etc/cron.d/*.bak"

---

Tip when using Laravel Scheduling on AWS Elastic Beanstalk!

Since one of the key features of Elastic Beanstalk is that it can autoscale and add more servers when needed, you might want to have a look on the new feature in Laravel Scheduling: Running Tasks On One Server.

In many cases you don't want your cron job to be executed on more than just one server. For example if you have a scheduled command for sending emails you don't want those sent multiple times.

NOTE: This requires that you use memcached or redis as your cache engine, as stated in the documentation. If you don't, have a look at the AWS service Elasticache.

NOTE 2: When using onOneServer() you must give the scheduled task a name using the name() method (before calling onOneServer()). Like so:

$schedule->command('my:task')
    ->name('my:task')
    ->daily()
    ->onOneServer();

Celin

Is NOTE2 really neccesary? – Reshape

Yes, at least when this was written. If I didn't name it, I got an error. Maybe this has changed in version 5.7 or 5.8, haven't checked. – Celin

This answer is really complete in every sense when it comes to Laravel & scheduling on AWS! Thank you for saving my lot of time! – Hamo

What is above remove_old_cron command section in ebextensions config ? Are there any .bak files generated by old crom which we are supposed to clean up ? Can you please elaborate usefulness of that ? – Hamo

@DhrumilBhankhar apparently when a new schedule_run gets created, the old one gets .bak suffix. remove_old_cron is to remove the old one (as the name suggests) – Gloucestershire

does this method still work? It is not working by me. – Cheder

Indeed nice and complete answer. Here is an AWS-DOCS example of a cron.config file from Github. – Frodin

THIS DOES NOT WORK! – Cheder

Maybe add this to the answer to make it work for Linux 2 instances. Linux 1 : . /opt/elasticbeanstalk/support/envvars Linux 2 : . /opt/elasticbeanstalk/deployment/env – Tahmosh

Answer 2

A simpler approach is to use the new Periodic Tasks feature. Using the .ebextensions for cron jobs may lead to multiple machines running the same job or other race conditions with auto-scaling.

Jobs defined in cron.yaml are loaded only by the Worker environment and are guaranteed to run only by one machine at a time (the leader). It has a nice syncing mechanism to make sure there's no duplication. From the docs:

Elastic Beanstalk uses leader election to determine which instance in your worker environment queues the periodic task. Each instance attempts to become leader by writing to an Amazon DynamoDB table. The first instance that succeeds is the leader, and must continue to write to the table to maintain leader status. If the leader goes out of service, another instance quickly takes its place.

Creating a Cron for a Single or Multiple Workers

Place cron.yaml in the root of the project:

version: 1
cron:
  - name: "schedule"
    url: "/worker/schedule"
    schedule: "* * * * *"

One thing to take into consideration is that in Beanstalk periodic tasks are designed to make an HTTP POST request to a URL in your application that in turn triggers the job you want to run. This is similar to how it also manages queues with SQS.

For Laravel

For Laravel specifically, you may create the routes and controllers to handle each scheduled job. But a better approach is to use Laravel's scheduler and have a single route that you call every minute.

This package will create those routes automatically for you https://github.com/dusterio/laravel-aws-worker

Troubleshooting Permissions

If you are running into trouble with the DynamoDB create Leader Table permissions when triggering a deploy from CodePipeline, it's because the CodePileline service role needs dynamodb:CreateTable. For instructions check these StackOverflow Question

Official Elastic Beanstalk Periodic Tasks Docs

Contradiction

Answer 3

you can use this with amazon Linux 2 using .ebextensions configurations you can run the commands directly

first you need to configure the command in separate file create file under .ebextensions called cron_job.txt and add this line

* * * * * root . /opt/elasticbeanstalk/deployment/env && /usr/bin/php /var/www/html/artisan schedule:run 1>> /var/www/html/laralog.log 2>&1

notice that the first part different from amazon Linux 2 from amazon Linux 1

it loads the environment variables:

Linux 1 : . /opt/elasticbeanstalk/support/envvars
Linux 2 : . /opt/elasticbeanstalk/deployment/env

and after initialing the command in this separated file

we need to fire it through the init.config file which have the container commands in .ebextensions

we can define it as follow :

container_commands:
   03cronjob:
        command: 'cat .ebextensions/cron_jobs.txt > /etc/cron.d/cron_jobs && chmod 644 /etc/cron.d/cron_jobs'

and that's it you can try it and find the cron jobs executed successfully.

and you can also read this explained article https://medium.com/qosoor/the-ultimate-guide-to-setup-cron-jobs-with-laravel-elastic-beanstalk-d497daaca1b0

Hope this is helpful

Awhile

does this actually work for anyone??? – Cheder

Answer 4

In AWS ECS we can use this without adding cron in to the container

https://github.com/spatie/laravel-cronless-schedule

This is how you can start the cronless schedule:

php artisan schedule:run-cronless

Hispaniola

Answer 5

This is for docker users, had some trouble with this so thought its worth posting.

The cron needs to be added to schedule_run file on the server. However, even if you add a container_name to the Dockerrun.aws.json file it changes it to that plus some extra information and therefore you cannot use the normal service name to run the cron.

So using $(docker ps -qf name=php-fpm) where name is part of the name of your container it will return the ID of the container. My container is called php-fpm.

Here is my working file (.ebextensions/01-cron.config).

files:
"/etc/cron.d/schedule_run":
    mode: "000644"
    owner: root
    group: root
    content: |
        * * * * * root docker exec -t $(docker ps -qf name=php-fpm) sh -c "php artisan schedule:run" >> /var/log/eb-cron.log 2>&1

commands:
    002-remove_old_cron:
        command: "rm -f /etc/cron.d/*.bak"

Note: it mite be that the first time this cron runs the container is not up. Since the cron in my example is running each minute it didnt matter too much as by the time it runs the 2nd time, the container is up and working.

Vision

Answer 6

The cron is not triggered because the location of the env file in the old solutions is wrong. Actually, there is nothing wrong with anything. Below is the command you can use currently. We use it in all projects.

Create a cronjob.config in the .ebextensions folder. Then put these to the inside of the file.

files:
    "/etc/cron.d/schedule_run":
        mode: "000644"
        owner: root
        group: root
        content: |
            * * * * * root . /opt/elasticbeanstalk/deployment/env && /usr/bin/php /var/www/html/artisan schedule:run 1>> /var/www/html/storage/logs/laravel_cron.log 2>&1

commands:
    remove_old_cron:
        command: "rm -f /etc/cron.d/*.bak"

Chrotoem

Answer 7

The accepted answer (of @Niklas) unfortunately didn't work for me.

There's even a more comprehensive explanation of that answer here:

https://medium.com/qosoor/the-ultimate-guide-to-setup-cron-jobs-with-laravel-elastic-beanstalk-d497daaca1b0

But as I said, that didn't work for me.

What worked for me is simpler: (.ebextensions/cron.config)

files:
    "/etc/cron.d/mycron":
        mode: "000644"
        owner: root
        group: root
        content: |
            * * * * * root /usr/local/bin/myscript.sh

    "/usr/local/bin/myscript.sh":
        mode: "000755"
        owner: root
        group: root
        content: |
            #!/bin/bash

            date > /tmp/date
            cd /var/www/html/ && php artisan schedule:run >> /dev/null 2>&1

            exit 0

commands:
    remove_old_cron:
        command: "rm -f /etc/cron.d/mycron.bak"

I simply copied the command from Laravel documentation and put it into the config file given by AWS documentation.

Herr

did this work for you? I can't get this to work. – Cheder

@Cheder Yes, it worked form me; I just checked, this file still exists and it's the same, it's working and triggering my scheduled artisan commands every day (night). Sorry for late reply. – Herr

Answer 8

I did the same thing but still, it's not working. Please find below code in file(.ebextensions/cron.config)

files:
    "/etc/cron.d/schedule_run":
        mode: "000644"
        owner: root
        group: root
        content: |
            55 23 * * * root . /opt/elasticbeanstalk/deployment/env && /usr/bin/php /var/www/html/artisan schedule:run 1>> /var/www/html/storage/logs/laravel_cron.log 2>&1

commands:
    remove_old_cron:
        command: "rm -f /etc/cron.d/*.bak"

I checked log in log file and I got below error.

**In BroadcastManager.php line 184:

Trying to access array offset on value of type null**

I can see below method in BroadcastManager.php line 184

protected function resolve($name)
    {
        $config = $this->getConfig($name);

        if (isset($this->customCreators[$config['driver']])) {
            return $this->callCustomCreator($config);
        }

        $driverMethod = 'create'.ucfirst($config['driver']).'Driver';

        if (! method_exists($this, $driverMethod)) {
            throw new InvalidArgumentException("Driver [{$config['driver']}] is not supported.");
        }

        return $this->{$driverMethod}($config);
    }

I checked it in local and I got 'pusher' value in $name variable. But I think it will not work at server.

I don't understand reason why it's not working. I think cron is working but error is coming from my command file.

namespace App\Console\Commands;

use Illuminate\Console\Command;

use App\Models\User;
use Illuminate\Support\Facades\Log;
use Carbon\Carbon;


class RemiderEmailToUsers extends Command
{
    /**
     * The name and signature of the console command.
     *
     * @var string
     */
    protected $signature = 'RemiderEmailToUsers';

    /**
     * The console command description.
     *
     * @var string
     */
    protected $description = 'Remider email to users';

    /**
     * Create a new command instance.
     *
     * @return void
     */
    public function __construct()
    {
        parent::__construct();
    }

    /**
     * Execute the console command.
     *
     * @return int
     */
    public function handle()
    {
        $users = User::whereDate('created_at', Carbon::today()->subDays(1))
         ->get()->toArray();

        $count = 0;

        if(!empty($users)) {
            
            echo "Total {$count} users added \n";
        }
        else {
            echo "No user available \n";
        }
      
    }
}

I am calling this command from kernel file.

namespace App\Console;

use Illuminate\Console\Scheduling\Schedule;
use Illuminate\Foundation\Console\Kernel as ConsoleKernel;

class Kernel extends ConsoleKernel
{
    /**
     * Define the application's command schedule.
     *
     * @param  \Illuminate\Console\Scheduling\Schedule  $schedule
     * @return void
     */

    protected $commands = [
        // Add your command here
        //'App\Console\Commands\RemiderEmailToTrialUsers',
        Commands\RemiderEmailToUsers::class,
    ];

    protected function schedule(Schedule $schedule)
    {
        $schedule->command('RemiderEmailToUsers')
                 ->dailyAt('23:55');
   
    }

After proper investigation, I got issues as a cron file is not getting the db host, pusher name, and other env variables values. So I created one .sh file in the .platform folder and put the below command

sudo cp /opt/elasticbeanstalk/deployment/env /var/www/html/.env

Rebroadcast

Answer 9

After several efforts, I found an alternative way to run a cron job easily. You can run a cron job easily in 3 Steps.

Step 1: Create a route

routes/web.php

Route::get('/cron/run',[HomeController::class, 'cron'])->name('cron');

Step 2: Create a function in the HomeController

public function cron()
{
    \Artisan::call("schedule:run");
    return 'run cron successful';
}

Step 3:

Run the URL every minute using https://cron-job.org/en/

Glasscock

This is a land mine that exposes your command to the public. – Dinny