shrink (truncate) file from beginning on linux

Question

Is it possible in Linux (and/or on other Unix) 'shrink' file from beginning? I'd like to use it for persistent queue (no existing implementation suits my needs). From end of file I guess it's possible with truncate().

Parlando

I started googling 'truncate beginning' after I wrote this post and it seems it isn't possible. – Parlando

Answer 1

If you are using ext4, xfs or some other modern file system, since Linux Kernel 3.15 you can use:

#include <fcntl.h>

int fallocate(int fd, int mode, off_t offset, off_t len);

with the FALLOC_FL_COLLAPSE_RANGE flag.

http://manpages.ubuntu.com/manpages/disco/en/man2/fallocate.2.html

Fractocumulus

Answer 2

You can try dropping half of logs using ex, but it is not as fast as I would like (5GB of logs takes ages):

ex -s -c "1d$(( $(wc -l /var/log/messages | awk '{ print $1 }') / 2 ))|x" /var/log/messages

Nessim

Answer 3

Yes, you can use cut or tail to remove portions of a file.

cut -b 17- input_file
tail -c +17 input_file

This will output the contents of input_file starting at the 17th byte, effectively removing the first 16 bytes of the file. Note that the cut example will also add a newline to the output.

Radiculitis

I was interested in efficient in-place update of file through some system call. What you suggest is not it. – Parlando

In that case, you could look at the system calls that the above commands are doing via strace and only do the 'meat' of the operation and not all the command line parsing, etc. – Radiculitis