how to find the maximum limit of Memory Allocation in c

Question

I want to determine what is the maximum limit of memory I can allocate in my computer. This is the code that I have written for this task:

#include <stdio.h>
#include <stdlib.h>

int main() {
    int j;
    int *primes;
    int i ;

    int limit = 2147483647;

    primes = malloc(sizeof(int) * limit);
    for (i = 0; i < limit; i++)
    {
        primes[i] = 1;
    }
    return 0;
}

How can I determine how much memory can be allocated without hit and trial? I have allocated the maximum int size in this example. But the program crashes. How much memory is really being allocated in this example?

Hayrick

are you on a embedded hardware? If so, behaviour is not predictable, see #22423233 – Orchid

First thing, your code is wrong. You did not check for the success of malloc(), so you're not achieving what you want. "how much memory is really being allocated in this example?" --> probably none. – Determine

Your code does not what you want. You should allocate memory chunks of say 1Mb in a loop, until mallocreturns NULL. Now you know (more or less) how much memory you can allocate on your system. – Blameless

@MichaelWalz True, but that involes "hit and trial", in'nit? – Determine

Also, you need to be aware that some modern operating systems overcommit, that is, they (seem to) allocate more memory than is available, as physical memory and swap space. For example, my Ubuntu system has 32 GB physical memory and just 4 GB swap space, so it can't possible have more than 36 GB of "actual virtual" memory, but malloc happily allocated 131071 GB (yes, 131 thousand gigabytes), and the program was then killed by the system when it had filled 10 of these gigabytes. – Dicotyledon

@SouravGhosh yes, but it's probably what the OP originally intended to do. – Blameless

XY problem. Why do you want to know? Even if there was a function to get the current memory available (physical? virtual?), it could already be wrong before that function returns to your code on a multiprocessing system. – Carrara

Sidenote: 1 is not a prime number. – Carrara

Answer 1

malloc() is allowed to fail, in which case it returns a NULL-pointer without allocating any memory. It is always an all-or-nothing allocation. It either succeeds and allocates a full chunk of memory of the requested size, or it fails, returning a NULL-pointer without allocating a single byte.

As for knowing how much memory is available - that really depends on the environment: what OS are you running this in, is that a 16-, 32-, 64-bit memory architecture?

For instance, if you are running on Windows 10 you can use the GlobalMemoryStatusEx() facility (refer to MSDN:GlobalMemoryStatusEx() for details).

Linux, OTOH, provides a neat sysconf() way of retreiving similar information. Refer to this page for more details.

Even if your OS is 64-bit that wouldn't necessarily mean that your application can access more than a certain limit. For instance Windows 7 64-bit will only let you address up to 8GB of memory in your application code even though the full virtual memory space is 16TB.

Czarevitch

What use would any value returned of such a function have on an OS where before the function even returns another process could allocate just that amount of memory? And typically one has no choice how much memory is required. If that is for buffers: leave that to the OS and libc. They most times know better anyway. (it might be different for very special software, but that would not be written by a beginner anyway) – Carrara

The GlobalMemoryStatusEx() returns (in ullAvailVirtual) the amount of memory available to the user process. That memory would not be trumped by other user-mode applications as they are isolated and don't eat into each other's memory space. See my comment about Windos 7 in the answer above (towards the end) – Czarevitch

That would imply that it is not guaranteed to be allocatable under all circumstances. Unless Windows was the first true clairvoyant OS. Without that my comment still applies. (the prefix "Global" for a user process limit is quite missleading, btw.) It is not about memory spaces (would be indeed strange if two processes have the same memory space - shared memory excluded). But maybe that is some DOS-extender think with "real-time memory double" as widely spread in the late 80ies/early 90ies. But those were just snake-oil. – Carrara

Nothing is ever guaranteed (except death and taxes), that is why programs dedicate a bigger part of their code to error handling – Czarevitch

You missed the point: if the value from that function has no relevance for practical use, why query it at all? After all, OP very likely has an XY-problem. – Carrara

malloc() does clearly NOT always return a NULL pointer when it fails, see #22423233 – Orchid

Answer 2

You code is wrong for so many reasons like

• You're assuming the max size (for the platform / environment) which is least likely to be true, less being portable.

• You're assuming that malloc() offers partial allocation (up to the available memory) which is again, wrong.

  Quoting the C11 standard, chapter §7.22.3.4, (emphasis mine)

  The malloc function returns either a null pointer or a pointer to the allocated space.

  So, it's either complete success (with the exact amount of requested memory allocated) or complete failure (return of null pointer), there's no trade-off like partial success with available memory or whatever which you might have assumed.

• You're not checking for malloc() success, resulting probable null-pointer dereference.

• You have no means to actually check for the success/ failure of allocation

I believe, you're in need of getrlimit() and family to achieve your purpose. Particular point of interest would be RLIMIT_DATA as the resource value.

That said,

1. "I have allocated the maximum int size in this example"

This does not seem to be connected with the limits for malloc() and family, anyways.

2. "But the program crashes"

That is most probably the result of undefined behavior. In your code, you directly dereference the returned pointer, without success check for malloc(). It is likely that malloc() has failed and returned a NULL, dereferencing that causes the UB.

Determine

Also using int limit = 2147483647;. This can be replaced by INT_MAX from limits.h. – Henequen

@Henequen but that's not related to this problem in hand, right? these are different limits and this particular value is of no interest here, as I see it. – Determine

I agree, just something the OP can consider I guess. Just thought I'd mention it since you are dot pointing issues in the OP's code. – Henequen

@Henequen Just to be clear, I did not mean to contradict you, i'm just clarifying that even after that change, the code would not make much sense, either. Appended my answer. :) – Determine

@Henequen but this code is working when i set limit less than 1500000. – Hayrick

yes you are right i have checked that malloc returned NULL pointer so please let me know how could i solve this problem as i need memory of 2147483647*4 bytes – Hayrick

@AzamAli thing is, if you already know the size of the allocation, wh you want to use malloc(), at all? Create an array and use compile time memory. In general, runtime memories are allocated from a scarce pool (i.e, heap) so they do not enjoy the privilege of allocating in huge amount, per se . – Determine

@RoadRunner: INT_MAX yields an int, not a size_t as malloc expects. It is also not guaranteed the max. available memory, nor the same as SIZE_MAX. – Carrara

@SouravGhosh: A statically allocated array of that size with a non-zero initialiser likely results in a very big executable. And if dynamic allocation fails for that size, most likely static will fail as well. Typically dynamic memory is very well meant for allocations as large as static. But the actual size depends on the platform. Maybe OP uses a 32 bit platform or has not enough virtual memory (although that would not necessarily result in malloc failing). – Carrara

@SouravGhosh i will take size from the user but first i want to see that is code is working for this max range if it will fix then i will move forward. – Hayrick

@Olaf i have tried static array but it also fails so what should i do next? – Hayrick

@SouravGhosh i'm using win 10 64bit – Hayrick

@Olaf Right indeed. Frankly speaking, if I need to use an array of that size, I'll probably first try to refactor my algorithm / approach. Any practicable code might not need that big array, in, well, 95% of the cases, so to speak. – Determine

@AzamAli: I'm not a clairvoyant. Maybe you try a better OS, more memory, etc? – Carrara

@SouravGhosh: I'd say a beginner indeed should reconsider the implementation, mmaping a file might be the better way. But if there was no need for such large array, there were no computers with 1TiB RAM. Even on normal notebooks, 64GiB RAM is no problem at all. After all, 2GiB*4 is just 8GiB - That's what mid-range graphics cards easily have. – Carrara

@Olaf Very nice spot. I guess I learnt something new today :) – Henequen

Answer 3

The maximum amount of memory you can allocate is controlled by several factors and can change over time.

These include: 1. Hardware limits 2. OS Limits 3. System Parameters 4. Process Quotas 5. Page file space

In addition, malloc is a very poor way to allocate large blocks of memory.

Your program crashes because malloc is returning null and you are using that return value without checking.

Marqueritemarques

i have checked that malloc has returned NULL you are right so please let me know what should i do to allocate large block if malloc is very poor for this – Hayrick

Answer 4

There are so many iteration perform by loop,you design that is the main causes of your program crash or loop dead infinity.

Answer to what you expect to know is very complex because of some key notes---->>

1. It depends on the platform that the program is working on like windows, linux or mac. I THINK that the amount of memory is not limited by anything, but physical memory.

Fact-> Although physical memory is Might be extended by Virtual memory, Not all platforms has feature of 'virtual memory'. C has no concept of virtual memory. Malloc allocates contiguous memory (meaning side by side or together in ram).
So, it depends on the way platform handles the request. It depends upon the C implementation.

2. The largest number (in bytes) representable by standard type 'size_t' (declared by ). This value can and does vary among implementations. Note that this value isn't necessarily as large as the host(i.e. end user) platform's available memory.

QUES. Is there any limitation on that? Where am I supposed to get this kind of information?

Ans. Malloc's argument is a size_t and the range of that type is [0,SIZE_MAX], so the maximum you can request is SIZE_MAX, which value varies from implementation to implementation and is defined in .

Note:- Whether a request for SIZE_MAX bytes will succeed depends on factors outside of the scope of this group.

Haughay

Mension nt Bhaijan ;) If it is really helpful @AzamAli then you may upvote the ans if you really want to ,so that it might be helpful for others as like you. :) – Haughay