Laravel DB Insert Error: Allowed Memory Size Exhausted

Question

I'm running into an issue when trying to insert ~20K records into my DB. I notice that even though I'm echoing inside my foreach loop, I'm not getting anything outputted in the command line. Instead, I get an error after inserting ~9440 records relating to...

Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 91 bytes) in /Users/me/Sites/Laravel/database/connection.php on line 293

Here is my code (tried using both Eloquent and Fluent):

<?php

class Process_Controller extends Base_Controller
{
    public function action_migrate()
    {
        $properties = DB::table('raw_properties')->get('id');
        $total = count($properties);

        foreach ($properties as $x => $p) {
            $r = RawProperty::find($p->id);
            $count = $x + 1;

            $prop_details = array(
                'column' => $r->field,
                // Total of 21 fields
            );

            DB::table('properties')->insert($prop_details);

            echo "Created #$count of $total\n";
        }
    }
}

Peterpeterborough

Answer 1

This error depicts that your PHP script has exhausted memory limit due to insufficient memory allocated for script.

You need to increase memory_limit using the ini_set function e.g ini_set('memory_limit','128M');

Amortizement

I just hit this problem to, increasing the memory limit is a quick fix. A much better solution is to turn off query logging as recommended by @Vergeboard below – Almemar

Answer 2

The accepted answer is fixing the symptom rather then the problem. The problem is the Laravel query log (in memory) is eating all your RAM when you execute such a large # of queries. See the answer here: https://mcmap.net/q/614424/-laravel-eloquent-memory-leak-retrieving-the-same-record-repeatedly

Or, in brief, turn off query logging via:

DB::disableQueryLog()

Before executing 20k queries

Vergeboard

I agree. While inserting 1M lines in a table 2G wasn't enough, but DB::disableQueryLog() fixed it for me. – Bridoon

thanks I'd def prefer to fix the problem rather than the symptom – Desmarais

Does this have to be enabled again to have Querylogging or is this for the current script? – Whited

In laravel 5 its disabled by default. So if you have memory problem you must do the opposite to inspect generated queries : DB::enableQueryLog(); [...] dd(DB::getQueryLog()); https://mcmap.net/q/126917/-how-to-get-the-query-executed-in-laravel-5-db-getquerylog-returning-empty-array – Liew

Answer 3

I did the DB::disableQueryLog() and continued to get the error. I ended up Pausing Telescope from recording the queries. You can do this from the telescope web interface > queries > Click the Pause Icon.

Cowlick