How do I memory map tmpfs files in sbcl?

Question

Exactly as the question says. I want to use shared memory to communicate between two lisp processes. Any pointers on how to do that?

I can see some tutorials on doing this in clozure at :-

http://ccl.clozure.com/manual/chapter4.7.html

Can someone point me to a similar library to do this with sbcl?

Astrograph

Thanks for the answers! They both seem to solve my problem so I'm going to try them out and accept the one I end up using. – Astrograph

Now see a portable mmap library: github.com/Shinmera/mmap – Sadirah

Answer 1

For a portable implementation, you might want to use the osicat library, which provides a CFFI wrapper for many POSIX calls in the osicat-posix package.

There is a very nice and short article with code for using it at http://wandrian.net/2012-04-07-1352-mmap-files-in-lisp.html (by Nicolas Martyanoff).

To preserve that, I mostly cite from there:

Mapping a file is done by opening it with osicat-posix:open, reading its size with fstat, then calling mmap. Once the file has been mapped we can close the file descriptor, it’s not needed anymore.

(defun mmap-file (path)
  (let ((fd (osicat-posix:open path (logior osicat-posix:o-rdonly))))
    (unwind-protect
         (let* ((size (osicat-posix:stat-size (osicat-posix:fstat fd)))
                (addr (osicat-posix:mmap (cffi:null-pointer) size
                                         (logior osicat-posix:prot-read)
                                         (logior osicat-posix:map-private)
                                         fd 0)))
           (values addr size))
      (osicat-posix:close fd))))

The mmap-file function returns two values: the address of the memory mapping and its size.

Unmapping this chunk of memory is done with osicat-posix:munmap.

Let’s add a macro to safely map and unmap files:

(defmacro with-mmapped-file ((file addr size) &body body)
  (let ((original-addr (gensym "ADDR-"))
        (original-size (gensym "SIZE-")))
    `(multiple-value-bind (,addr ,size)
         (mmap-file ,file)
       (let ((,original-addr ,addr)
             (,original-size ,size))
         (unwind-protect
              (progn ,@body)
           (osicat-posix:munmap ,original-addr ,original-size))))))

This macro mmaps the given file and binds the two given variables to its address and and size. You can then calculate address pointers with cffi:inc-pointer and access the file contents with cffi:mem-aref. You might want to build your own wrappers around this to represent the format of your file (e. g. plain text in UTF-8).

(In comparison to the posting linked above, I removed the wrapping of osicat-posix:munmap into another function of exactly the same signature and effect, because it seemed superfluous to me.)

Coincidentally

Note: osicat-posix:mmap NOT support Windows. There is CreateFileMapping on Windows, please check SBCL source code file src/code/win32.lisp. – Willow

Answer 2

There is low-level mmap function bundled with sbcl:

CL-USER> (apropos "MMAP")
SB-POSIX:MMAP (fbound)
; No value
CL-USER> (describe 'sb-posix:mmap)
SB-POSIX:MMAP
[symbol]

MMAP names a compiled function:
 Lambda-list: (ADDR LENGTH PROT FLAGS FD OFFSET)
 Derived type: (FUNCTION (T T T T T T)
                (VALUES SYSTEM-AREA-POINTER &OPTIONAL))
 Inline proclamation: INLINE (inline expansion available)
 Source file: SYS:CONTRIB;SB-POSIX;INTERFACE.LISP.NEWEST
; No value

You have to use explicit address arithmetics to use it, as in C.

Caffeine