Upper memory limit for PHP/Apache

Question

I'm getting the error when I run my PHP script....

Fatal error: Out of memory (allocated 1827405824) (tried to allocate 88800 bytes)

I've added this line to my PHP script..

ini_set("memory_limit","3000M");

This statement does seem to correctly control the memory usage, but I dont seem to be able to get it above about 1.8GB. Its as if the upper memory limit is being restricted somewhere else. I've also added to the php.ini...

memory_limit = 3000M

Does anyone know if the memory is restricted elsewhere?

I'm running a local server with Xampp. I have Windows 7, 64-bit with 4GB RAM. My script uses PHP's GD image library and I get the error when trying to allocate an image reference with ImageCreateTrueColor().

(I know this is a huge amount of memory - but this is just a one-of script, and its just a lot easier to do it this way.)

Thanks.

Update....

@elusive @Orbling I expect everybody's bored whith this question, but here is the simplified code which illustrates the problem.

<?php
    ini_set("memory_limit","4000000000");
    echo "ini_get = " . ini_get('memory_limit') . "<br>\n";
    echo "memory_get_usage = " . memory_get_usage(true) . "<br>\n";
    $bigImageHandle = imagecreatetruecolor(22200, 24800);  //this is line 5
?>

Browser output...

ini_get = 4000000000
memory_get_usage = 524288

Fatal error: Out of memory (allocated 1843396608) (tried to allocate 88800 bytes) in
E:\User\My_Webs\experiments\houseshunting\temp\osMaps\t1.php on line 5

I tested this out with a smaller set of tiles and the memory used by imagecreatetruecolor() and I estimate I need 2.7GB

Issy

Are you sure that you need such a huge amount of data at the same time? Are you composing an image out of thousands of other images? – Varicella

I agree with elusive. You are almost certainly doing something wrong to need that much mem. – Twister

I am making a large image out of many small images. To do this I need to allocate a large image reference - which is causing the problem. Or are you suggesting that there is a way to combine many images together that doesnt require a large amount of memory? – Issy

@spiderplant0: AFAIK, GD copies the contents of one of the small images onto your big image. You should be able to free the memory of the small image after you copied it. You can do this one image at a time and reduce the overall memory that is used simultaneously. The large image itself is no problem, since it is only about 88KB in memory. The problem is the (obviously) massive amout of small images that are all in memory at the same time. – Varicella

elusive, the structure of my code is: (1) read filenames of all small images into an array. (2) calculate size of big image and create a big image (imagecreatetruecolor). (3) open each small image in turn and merge into the big image (imagecopymerge). The memory usage after step 1 is only 1048576 Bytes. I get the memory fault during the creation of the big image - i.e. well before opening any of the small images – Issy

@spiderplant0: PHP states that you already allocated 1,7 gigs before your big image's 88K bytes. Are you accidentally looping anything here? It would be best if you'd show some code. By the way: If you want to reply to someone, add an @ in front of the users name. That ensures that he is notified. Otherwise, only the author of the commented post gets notified. – Varicella

@Issy With regard to step 2. This calculation of the large image size, how is that done, do you get size info from the smaller images perhaps? – Paratroops

@elusive Ok I'll post my code in another 'reply' rather than a comment. Dont know what the best way to do this is - I dont really want to answer my own question? – Issy

@elusive - its as if imagecreatetruecolor() allocates memory in small steps because as far as I can see all 1.7GB is being allocated by imagecreatetruecolor. $bigImageHandle = imagecreatetruecolor($tileWidth * $bigImageTilesX, $tileHeight * $bigImageTilesY); – Issy

@spiderplant0: That is a huge image! Assuming 8 bytes per pixel (RGBA: 22200 x 24800 x 8 = 4200 MB), that is more than 4 gigs of memory and is not going to work. – Varicella

@elusive Actually I believe its more like 4 bytes per pixel. 2.7GB. I know its a lot of memory but i calculate it will take 30 min to run and its just a one of and the alternative is it split it up which will be a pain. – Issy

@Issy Not possible with x32 PHP, it is beyond its memory space limitations. – Paratroops

@spiderplant0: Are you sure that you want to do this? Nobody will be able to even view this image, since it is so huge. It might be smaller than i assumed, but that does not change the fact that this is a massive amount of data. How do you intend to use this image? Are you sure that you need these tiles full-size? – Varicella

@elusive Once the big image is created it gets sent to GIS software for map reprojection, then it gets split into 256*256 google map tiles. Because the reprojection cause the image to become 'non-square' it will be difficult to split this into stages and then try to create the tiles. And even if I split it up the images will still be quite big. – Issy

@Issy Difficult or impossible? Because I think you will find you have to, the data quantities are just too big to do in one move. – Paratroops

Answer 1

Using Acquia Dev Desktop, I had many memory limit crashes.

After having increased the memory limit into PHP.ini.

php_value memory_limit                  1024M
php_value max_execution_time            3000

This issue was less frequent but still occuring ( Especially with Feature Recreate )

Into my httpd.conf I increased the StackThread to 16M

ThreadStackSize 16*1024*1024

And it solved the memory crash issue. Hope it can help

Gabrielgabriela

Seems to solve my problem (Win7, 64bit, xampp) for this error message: VirtualAlloc() failed: [...] VirtualFree() failed: [...] [...]] PHP Fatal error: Out of memory (allocated 98566144) (tried to allocate 16384 bytes) in [...] – Misspell

Answer 2

You're running on a 64-bit operating system, but Apache and PHP are likely still 32-bit. If you're using mod_php, apache would be the limiting factor here.

32-bit processes are limited about 2GiB of RAM unless you used the /3GB switch and the software is aware of 3GB support.

That still leaves up about 200 MiB that seems unused, but its small enough that it can be used by various libraries that all have to be loaded in memory

As far as I know, the library usage won't show up in the committed memory, but still counts towards the 2GiB limit (much like device memory counts towards the 4GiB limit on 32-bit windows. Where installing 2 GiB graphics card brings you down to under 2GiB of usable RAM).

Most likely solution? Install a 64-bit PHP, and then dispatch it to that (using a system() call, perhaps)

Aubree

Answer 3

In Ubuntu 18.04
Check version PHP: php -v
In my case I have PHP 7.4
Edit file: nano /etc/php/7.4/apache2/php.ini
Search and change memory_limit = 2048M
Edit file: nano /etc/php/7.4/cli/php.ini
Search and change memory_limit = 2048M
Finally: systemctl restart apache2

Merodach

Answer 4

Which PHP version are you using?

The memory_limit variable is, or was, contained in a 32-bit integer, so can not go above 2GB.

See: http://bugs.php.net/bug.php?id=39132&edit=1

From the bottom comment on that bug report, it might be the routine that translates the human readable form to a number, try putting it in digits.

Paratroops

I assume you have the available memory? Have you tried setting it in the php.ini instead? – Paratroops

Yes I tried it in both places. I have 4GB physical RAM. According to the task manager 44% is in use before running script (though its dificult to say with windows superfetch). Though I thought it would just use pagefile cache if it ran out anyway? – Issy

Hmm, well if it turns out the memory is going to be an issue. What are you attempting to do with the image? That error message suggested you had already allocated about 1.8GB of RAM within PHP. – Paratroops

Only 1048576 Bytes has been allocated prior to creating the big image. See reply to elusive above. I tried the script with a smaller set of images and it works fine. – Issy

@Issy Are you running a 64bit version of PHP? – Paratroops

I assumed I should be allowed 4GB by PHP. I dont understand why it gives an error after only 1.7GB. 1827405824 + 88800 < 4GB (and also way less than 2GB) – Issy

No. its 32-bit. I had a look for a 64-bit version of xampp but as far as I can see its not possible. And figuring out how to compile it etc is probably beyond me. Though if you have good reason this is the cause I will give it a go. Do you know the best way to go about this? – Issy

32-bit apps can not use 4GB of memory unless they are compiled to be aware of the extra available space (/LARGEMEMORYAWARE flag set, they are predominantly limited to 2GB per process. Then PHP itself will have a fair bit of overhead, which will probably account for the drop to 1.7-.8GB. – Paratroops

@erloewe: Aye, quite possibly - I figured the comments attached to the answer were ok, probably would be better if I edited them in to the answer, but it was a fair while ago now. – Paratroops

Answer 5

Check your Apache config (e.g., httpd.conf). There's likely an RLimitMEM directive limiting the memory allow for children processes handling requests.

So, you can set your PHP limit all you want, if Apache spawns the process with a memory limit, you can't exceed that.

If you're on a hosted service and have a shared server, likely you don't have access to this config and need to work with your provider. As you can see, it's configuration that applies server-wide... you're not likely going to get them to change this. Then again, if you're looking to spawn bigger than 1.5Gig processes, you prolly should be either solving the problem a different way (others have suggested this) or getting a dedicated server of some kind (e.g. EC2).

For example:

/usr/local/apache/conf
#RLimitMEM 85643200 104857600   # Limit to: 80Mb / process, 100Mb total
RLimitMEM 134217728 537395200   # Limit to: 128Mb / Process, 512Mb total

Paper

Answer 6

The issue is likely to be caused by running 32-bit apache and php. Try upgrading these to 64-bit binaries and see if that fixes the issue.

Petrifaction

Answer 7

I've had exactly the same problem and after searching a lot I discovered it had nothing to do wih memory limit but with a bug in my code: I had a function using an array, located in external file. In this function I had set the array as "global" but missed to include the file with tis array.... Like the spiderplant0's error, this was giving me an error with a very very huge memory allocation. So check your code and try to see which part create this error.

Undervest

Answer 8

Try this

set_time_limit(300);
ini_set('memory_limit', '20000M');

Crop

Did you notice that you're suggesting 20 gigs? How is this supposed to work? – Varicella

@Varicella it's supposed to work LIKE A CHARM 😁 – Perverted

Answer 9

Try this:

#php_value memory_limit 300M 
#php_value upload_max_filesize 200M 
#php_value post_max_size 200M 
#php_value max_execution_time 80000 
#php_value max_input_time 80000   

Peeling

Where do I put these lines? Internet search suggests its for .htaccess? I dont seem to have this file in the xammp folder? – Issy

@Pradeep Singh: Why did you put in the #, anyway? – Varicella

I dont have an .htaccess file where should I create this file – Issy

then In php.ini add upload_max_filesize=200M memory_limit=300M upload_max_filesize=200M post_max_size=200M max_execution_time=100000 max_input_time=100000 – Peeling

Thanks Pradeep Singh. But why the 300M etc? Did you mean 3000M? – Issy

@Pradeep Singh - pasted at bottom of php.ini and restarted server. No change. – Issy

Answer 10

try ini_set('memory_limit', '-1');

Wearisome