Using ini_set("memory_limit", "-1") and still out of memory

Question

I'm processing an old database php array to a new database. The data .php files are in total around 220 MB large.

I've inserted these lines in the script so that it should run fine:

ini_set("memory_limit", "-1");
set_time_limit(0);

This is how I include the products:

// The exports made by PHPMYADMIN, exported as PHP-Array
require_once 'export/tx_ttproductsv2_products.php';
require_once 'export/tx_ttproductsv2_keyword.php';
require_once 'export/tx_ttproductsv2_keywords_in_products.php';
require_once 'export/tx_ttproductsv2_typebook.php';
require_once 'export/tx_ttproductsv2_typegospel7.php';
require_once 'export/tx_ttproductsv2_typemedia.php';

When the script is trying to require them I get this error:

PHP Fatal error: Out of memory (allocated 880541696) (tried to allocate 469762048 bytes) in ......

I've got an local EasyPHP installation running on x64 Win 7 SP1. I've got 6 GB memory, Intel i5.

How can I get PHP to run the whole script without the memory error?

Isaiah

Try to increase upload_max_filesize & post_max_size ? – Steele

@KhawerZeshan, i don't upload things? Or am i seeing that wrong? – Isaiah

exports made by PHPMYADMIN is a file or what? – Steele

You should change the memory_limit in the php.ini file. You are however allready using 839.75 MegaByte. That is a lot... If the database is only 220MB large this means your script is creating huge amounts of overhead. Try procesing 1 file at a time instead of all at once – Coon

@KhawerZeshan, it's an file wich holds an PHP array with the colums... – Isaiah

@Pinoniq, does it make's an difrnce between the ini_set and the php.ini then? And i was hoping that it wasn't needed to do 1 step at a time... – Isaiah

@Isaiah Some configurations don't allow you to overwrite php.ini settings using ini_set. I'm not familiar with easyPHP or Windows so I'm not sure that my answer will work. You are however still using a lot of memory so instead of parsing all files at once, parse them one by one – Coon

@Pinoniq, I've used it before with less files, and that worked ( ran out of memory, added those lines and it was done ). And it is indeed an large overkill.. So i will parse them 1 by 1 them... A shame, but that's the reality... – Isaiah

Answer 1

Memory limitation comes from the OS, not from PHP itself.

Your script allocated 800MB and is trying to allocate further 500MB

Try to run the script on 64bit OS with 64 bit PHP.

Bateau

That would probably fix a lot, but i'm not able to test it now ( i need it working right away ). – Isaiah

The best solution is for you to fetch data and process it in chunks instead of loading all into the memory and then processing. – Bateau

I know, I've splitted it up in 3 parts ( coding right now ), and that should work out fine to... Thanks anywhay for the help and ideas ;) – Isaiah