Is there a difference between boost iostream mapped file and boost interprocess mapped file?

Question

I want to create a mapped binary file into memory; however I am not sure how to create the file to be mapped into the system. I read the documentation several times and realize there are 2 mapped file implementations, one in iostream and the other in interprocess.

Do you guys have any idea on how to create a mapped file into shared memory? I am trying to allow a multi-threaded program to read an array of large double written in a binary file format. Also what is the difference between the mapped file in iostream and interprocess?

Complimentary

Answer 1

As far as I can tell, iostreams will place the mapped file in shared memory (this is what you want); however, interprocess however places the file in a another process's address space.

You should probably use iostreams unless you have multiple processes (not threads) that will be communicating with each other in some way.

Yellowthroat

Thanks for the clarification Joe, I shall try out iostream. – Complimentary

@Joe D: that's not the impression I get from the interprocess library, at all. Specifically, from the documentation: "the system creates a file mapping to associate the file and the address space of the process." For multiple processes, it essentially shared memory with a filesystem backup. – Bio

Yes, but the interprocess library, doesn't create a filesystem backup, it simply maps the file in interprocess shared memory. However iostreams, will map it into the memory for a single process. Which is probably (I haven't checked the implementation) more efficient. – Yellowthroat

I agree with your conclusion - but I'm not sure that the description of interprocess is quite accurate. It allows - rather than forces - the same file to be loaded into other processes; it also hides the differences between pointers (with 'offset pointers') so that if the file is loaded at address A in one process and a different address B in another, the two processes can still access the data structures embedded in it cleanly. – Barbrabarbuda

What I meant by it was, that instead of allowing the file to simply be shared between threads, it would place it in the shared memory - which will allow it to be accessed by other processes. I agree with you that I made a mistake when I said it forces though. – Yellowthroat

I'm sorry, does that mean I should use interprocess for memory mapping? – Complimentary

Only if your using multiple processes – Yellowthroat

I'm trying to create an PHP extension, with opencv lib. I have a set of feature that would be used by the extension, and I'm trying to use the lib to load into the memory and shared by the php thread. – Complimentary

Anyone here tell me if raw mmap() would be more-efficient than Boost iostreams (even if only a small % faster)? – Bryology

Answer 2

The chief difference I see between the two is how they are used. In boost-interprocess, to use a memory mapped file, you create objects in that memory space using placement new, and those objects are automatically persistent in binary form in your file. Other processes can then map the same file and use those objects, or the program itself can use it as a persistent cache and reload them later. Memory mapped files in boost-iostreams act just like file streams, with the added benefits of being a boost::iostream, and would provide stream semantics to interprocess communication.

For a single process, there isn't much benefit to using boost::iostream memory mapped files. However, it can reduce the latency in working with the file as it will have already been loaded into memory. But, you only gain this benefit if you are constantly rewriting parts of the file. For a single read/write pass of the file, there may not be any speed up.

Bio