How much memory should 'managed_shared_memory' allocate? (boost)

Question

I'm looking for a definitive answer (if indeed one exists) on how much memory should be allocated when creating a static chunks of shared memory via boost::interprocess's managed_shared_memory. Even official examples seem to allocate arbitrarily large chunks of memory.

Consider the following structure:

// Example: simple struct with two 4-byte fields
struct Point2D {
  int x, y;
};

My initial reaction is that the necessary size would be 8 bytes, or sizeof(Point2D). This fails miserably when I attempt to construct an object, giving me seg-faults at runtime.

// BAD: 8 bytes is nowhere near enough memory allocated.
managed_shared_memory segment(create_only, "My shared memory", sizeof(Point2D));

What read/write operation is causing seg-faults? Stack operations? Temporarily allocations within segment.construct()? How much overhead is necessary when allocating shared memory?

By trial-and-error I found that multiplying the size by 4 can work for the above structure, but falls apart when I start adding more fields to my struct. So, that reeks of a bad hack.

Some might argue that "memory is cheap" in the modern PC, but I disagree with this philosophy and dislike allocating more than I need to, if I can avoid it. I dug around the Boost docs yesterday and couldn't find any recommendations. Here's to learning something new today!

Dorsoventral

People might disagree with me here, but I have never in my life coded along the lines of "memory is cheap". Buying memory is not necessarily expensive compared to how it used to be, but it is very much like money. The more you have, the more you spend. Every memory upgrade I've bought for my computer, I've maxed out pretty fast now that i can "run more stuff". I've always tried to code conservatively in this respect because it is not necessarily cheap for my application. Anyway, just my 2c on that :) – Cindacindee

I agree 100%! And that is the entire reason I am asking this question. I only threw that comment out there to dissuade anyone saying "who cares, just allocate 1k and be done with it." I'll try to make it more clear in the post. – Dorsoventral

Ah ok :) "Some might argue" is much better! – Cindacindee

Answer 1

From this paragraph of the documentation :

The memory algorithm is an object that is placed in the first bytes of a shared memory/memory mapped file segment.

Layout of the memory segment :

 ____________ __________ ____________________________________________  
|            |          |                                            | 
|   memory   | reserved |  The memory algorithm will return portions | 
| algorithm  |          |  of the rest of the segment.               | 
|____________|__________|____________________________________________| 

The library has an extra memory overhead sitting at the beginning of the segment, thus occupying a few bytes of your requested size. According to this post and this post, this exact number of additional bytes cannot be determined :

You can't calculate it, because there are memory allocation bookeeping and fragmentation issues that change in runtime depending on your allocation/deallocation pattern. And shared memory is allocated by pages by the OS (4K on linux 64k on windows), so any allocation will be in practice allocated rounded to a page:

    managed_shared_memory segment(create_only, "name", 20);

will waste the same memory as:

    managed_shared_memory segment(create_only, "name", 4096);

Kershaw

Good job finding that old post; I searched for quite a while and came up dry. So, by "indeed" do you mean that there is no concrete answer? That's what I get from Ion Gaztañaga's response... Also, thanks for that link to the Boost docs. The memory map ASCII art helps out, though I had no luck in programmatically determining memory algorithm + reserved. But ultimately, it's a moot point since my actual implementation stores 5-10 structure instances (also mentioned by Gaztañaga). Still, this seems like a valid question, even if it's mostly "academic." – Dorsoventral

Ah, there we go. I knew OS page size must be a relevant detail; the quote above confirms my suspicions. It looks like "best guess" allocation will have to do. Thanks again. – Dorsoventral

Answer 2

Something like using the OS'es memory page size works. In my case this works..

off_t size = sizeof(class1) + (sizeof(class2) * 3);
// round up to the OS page size.
long page_size = sysconf(_SC_PAGE_SIZE);
size = ((size / page_size) + (size % page_size ? 1 : 0)) * page_size;

Using boost::managed_shared_memory allows you to construct objects in the resulting space. Something like....

shared_memory_object::remove(m_name.c_str());
m_shared.reset(new managed_shared_memory(create_only, "myspace", size));
m_class1 = m_shared->construct<class1>("class1")();
m_class2 = m_shared->construct<class2>("class2")[3]();

Amando

+1 I like the idea of rounding up to system page size. However, it still looks like we're creating an arbitrary amount of padding. In this case "3" -- three times sizeof(class2). Am I correct? I've since moved on to other projects, but I'm still interested in the best way to allocate shared memory with the least amount of waste. – Dorsoventral

I can't see anyway to get around using the page size for the shared memory area. (Fixed a bug in the second bit of code.) However you can do the type of suballocation that I was using in the second bit of code inside of the shared memory area. look at en.highscore.de/cpp/boost/… gives a pretty good idea of the many ways that sub-allocation can be done. I'd better learn this editor so I can link more cleanly sigh – Amando