Accessing bytes beyond most-positive-fixnum in emacs lisp

Question

TL;DR: Can 32bit emacs-lisp byte ranges beyond the first 512 MB of a file?

I like using emacs-lisp for various data processing tasks due to its (obviously) high level of integration with the editor.

When thus writing custom functions I sometimes run into limits imposed by most-positive-fixnum. While I can live with and understand the reasons for not being able to load a whole 1GB file into a buffer, the insert-file-contents[-literally] interface also doesn't allow accessing chunks of data beyond the first 512 MB of a file, since it requires as argument an integer byte range.

Is there some way to circumvent this limitation? As far as I know currently insert-file-contents is the most low-level file reading function available to emacs-lisp, insert-file-contents-literally being just a variant of the first configured by setting a couple of variables, i.e. emacs-lisp doesn't seem to provide any low-level fopen type file interface.

Curiously at some point (possibly using a self-compiled binary) calling insert-file-contents-literally with floating point arguments worked for me, allowing access to high byte ranges, at the cost of the risk of rounding errors. Using the latest precompiled emacs version from ftp.gnu.org (24.3 as of writing) however, doing so yields a (wrong-type-argument file-offset NUMBER) error.

Ethben

Seems you have to use external commands like head and tail, see emacswiki.org/emacs/VLF. In that case you could fiddle around with emacs-calc arbitrary precision numbers. – Aesculapian

Using external commands sadly introduces platform dependence I want to avoid; I'm using emacs on Windows mostly. – Ethben

Answer 1

There is no way to do this solely in Emacs. You have to use some external program in some way. It's a known limitation, I believe it is in the bug tracker already.

Crossroad