Remove the last line from a file in Bash

Question

I have a file, foo.txt, containing the following lines:

a
b
c

I want a simple command that results in the contents of foo.txt being:

a
b

Oared

Answer 1

Using GNU sed:

sed -i '$ d' foo.txt

The -i option does not exist in GNU sed versions older than 3.95, so you have to use it as a filter with a temporary file:

cp foo.txt foo.txt.tmp
sed '$ d' foo.txt.tmp > foo.txt
rm -f foo.txt.tmp

Of course, in that case you could also use head -n -1 instead of sed.

MacOS:

On Mac OS X (as of 10.7.4), the equivalent of the sed -i command above is

sed -i '' -e '$ d' foo.txt

Tankersley

On Mac OS X (as of 10.7.4), the equivalent of the sed -i command above is sed -i '' -e '$ d' foo.txt. Also, head -n -1 won't currently work on a Mac. – Misspeak

Could you explain what '$ d' regex does? It's question about removing last line, so I think this is the most important part for everyone viewing this question. Thanks :) – Democracy

@Miro : By no means is $ d a regex. It is a sed command. d is the command for deleting a line, while $ means "the last line in the file". When specifying a location (called "range" in sed lingo) before a command, that command is only applied to the specified location. So, this command explicitly says "in the range of the last line in a file, delete it". Quite slick and straight to the point, if you ask me. – Naples

How do your remove the last line but only if it's an empty line? – Ramiroramjet

which versions of sed is the -i option compatible with? – Stutz

@Alex: updated for GNU sed; no idea about the various BSD/UNIX/MacOS sed versions... – Tankersley

When there are multiple empty lines at the end of the file, this seems to remove the last two empty lines instead of last one empty line. – Coreencorel

As @Ramiroramjet and Panu Haaramo note, this has problems with empty lines at the end – Mejia

Answer 2

This is by far the fastest and simplest solution, especially on big files:

head -n -1 foo.txt > temp.txt ; mv temp.txt foo.txt

if You want to delete the top line use this:

tail -n +2 foo.txt

which means output lines starting at line 2.

Do not use sed for deleting lines from the top or bottom of a file -- it's very very slow if the file is large.

Fractionate

head -n -1 foo.txt is enough – Cowboy

head -n -1 doesn't work on bdsutils' head. at least not in the version macos is using, so this doesn't work. – Lourdeslourie

@Lourdeslourie On Mac, you can brew install coreutils (GNU core utilities) and use ghead instead of head. – Incomprehensible

Here is a simple bash script that automates it in case you have multiple files with different number of lines at the bottom to delete: cat TAILfixer FILE=$1; TAIL=$2; head -n -${TAIL} ${FILE} > temp ; mv temp ${FILE} Run it for deleting 4 lines for instance from myfile as: ./TAILfixer myfile 4 of course first make it executable by chmod +x TAILfixer – Dillard

Thumbs up because it worked, but for all the comparison to sed, this command is pretty darn slow as well – Insular

this soulution is not working when the last line is empty as I see. It deleted the last non-empty line and left the last line which is an empty line. I actually wanted to delete that empty line but I failed with this. – Clop

Answer 3

For large files

I had trouble with all the answers here because I was working with a HUGE file (~300Gb) and none of the solutions scaled. Here's my solution:

filename="example.txt"

file_size="$(stat --format=%s "$filename")"
trim_count="$(tail -n1 "$filename" | wc -c)"
end_position="$(echo "$file_size - $trim_count" | bc)"

dd if=/dev/null of="$filename" bs=1 seek="$end_position"

Or alternatively, as a one liner:

dd if=/dev/null of=<filename> bs=1 seek=$(echo $(stat --format=%s <filename> ) - $( tail -n1 <filename> | wc -c) | bc )

In words: Find out the length of the file you want to end up with (length of file minus length of length of its last line, using bc), and set that position to be the end of the file (by dding one byte of /dev/null onto it).

This is fast because tail starts reading from the end, and dd will overwrite the file in place rather than copy (and parse) every line of the file, which is what the other solutions do.

NOTE: This removes the line from the file in place! Make a backup or test on a dummy file before trying it out on your own file!

Undercast

I didn't even know about dd. This should be the top answer. Thank you so much! It's a shame the solution does not work for (block) gzipped files. – Swope

Very, very clever approach! Just a note: it requires and operates on an real file, so it can not be used on pipes, command substitutions, redirections and such chains. – Piegari

This should be the top answer. Worked instantly for my ~8 GB file. – Brooch

mac users: dd if=/dev/null of=<filename> bs=1 seek=$(echo $(stat -f=%z <filename> | cut -c 2- ) - $( tail -n1 <filename> | wc -c) | bc ) – Punk

This approach is the way to go for large files! – Split

Can we test somehow if the last line is empty? – Apomixis

@Lanti: run tail -n1 on the file and take a look! – Undercast

I desperately scrolled through previous 2 answers that propose using sed and head+>, and YEAH, exactly what I needed, finally, thanks much! Must be the top answer, totally agree! – Benzvi

You can also calculate the number of bytes to remove with bash directly: end_position=$(($file_size - $trim_count)) – Arlin

Can confirm alternative calculation of the number of bytes is working. Was helpful to me because: -bash: bc: command not found (Ubuntu 20.04) – Ogpu

Answer 4

To remove the last line from a file without reading the whole file or rewriting anything, you can use

tail -n 1 "$file" | wc -c | xargs -I {} truncate "$file" -s -{}

To remove the last line and also print it on stdout ("pop" it), you can combine that command with tee:

tail -n 1 "$file" | tee >(wc -c | xargs -I {} truncate "$file" -s -{})

These commands can efficiently process a very large file. This is similar to, and inspired by, Yossi's answer, but it avoids using a few extra functions.

If you're going to use these repeatedly and want error handling and some other features, you can use the poptail command here: https://github.com/donm/evenmoreutils

Winnifredwinning

Quick note: truncate is not available on OS X by default. But it's easy to install using Brew: brew install truncate. – Highstepper

Very nice. Much better without dd. I was terrified of using dd as a single misplaced digit or letter can spell disaster.. +1 – Undercast

(JOKE WARNING) Actually, ("dd" --> "disaster") requires 1 substitution and 6 insertions; hardly "a single misplaced digit or letter". :) – Sass

Question, how is tail | wc not reading the whole file? – Vilberg

@Vilberg tail reads backwards from the end of the file looking for newlines.... – Undercast

Answer 5

On macOS, head -n -1 wont work but you can use this command:

cat file.txt | tail -r | tail -n +2 | tail -r

1. tail -r reverses the order of lines in its input

2. tail -n +2 prints all the lines starting from the second line in its input

Tacy

Installing coreutils using Homebrew will give you the GNU head as 'ghead', allowing you to do ghead -n -1 – Avian

Answer 6

Mac Users

if you only want the last line deleted output without changing the file itself do

sed -e '$ d' foo.txt

if you want to delete the last line of the input file itself do

sed -i '' -e '$ d' foo.txt

Promissory

This works for me but I don't understand. anyhow, it worked. – Camarata

The flag -i '' tells sed to modify file in-place, while keeping a backup in a file with the extension provided as a parameter. Since the parameter is an empty string, no backup file is created. -e tells sed to execute a command. The command $ d means: find the last line ($) and delete it (d). – Avoidance

Answer 7

echo -e '$d\nw\nq'| ed foo.txt

Perichondrium

For a filter solution that does not edit the file in place, use sed '$d'. – Perichondrium

Answer 8

Linux

$ means the last line, and d means delete:

sed '$d' ~/path/to/your/file/name

MacOS

Equivalent of the sed -i:

sed -i '' -e '$ d' ~/path/to/your/file/name

Covell

Answer 9

awk 'NR>1{print buf}{buf = $0}'

Essentially, this code says the following:

For each line after the first, print the buffered line

for each line, reset the buffer

The buffer is lagged by one line, hence you end up printing lines 1 to n-1

Jarid

This solution is a filter and does not edit the file in place. – Perichondrium

POSIX compiant and most cross-platform and cross-shell – Kaveri

Answer 10

Here is a solution using sponge (from the moreutils package):

head -n -1 foo.txt | sponge foo.txt

Summary of solutions:

1. If you want a fast solution for large files, use the efficient tail or dd approach.

2. If you want something easy to extend/tweak and portable, use the redirect and move approach.

3. If you want something easy to extend/tweak, the file is not too large, portability (i.e., depending on moreutils package) is not an issue, and you are a fan of square pants, consider the sponge approach.

A nice benefit of the sponge approach, compared to "redirect and move" approaches, is that sponge preserves file permissions.

Sponge uses considerably more RAM compared to the "redirect and move" approach. This gains a bit of speed (only about 20%), but if you're interested in speed the "efficient tail" and dd approaches are the way to go.

Continental

Answer 11

OK processing a good amount of data and the output was OK, but had one junk line.

If I piped the output of the script to:

| sed -i '$ d' I would get the following error and finally no output at all sed: no input files

But | head -n -1 worked!

Abert

Answer 12

Both of these solutions are here in other forms. I found these a little more practical, clear, and useful:

Using dd:

BADLINESCOUNT=1
ORIGINALFILE=/tmp/whatever
dd if=${ORIGINALFILE} of=${ORIGINALFILE}.tmp status=none bs=1 count=$(printf "$(stat --format=%s ${ORIGINALFILE}) - $(tail -n${BADLINESCOUNT} ${ORIGINALFILE} | wc -c)\n" | bc )
/bin/mv -f ${ORIGINALFILE}.tmp ${ORIGINALFILE}

Using truncate:

BADLINESCOUNT=1
ORIGINALFILE=/tmp/whatever
truncate -s $(printf "$(stat --format=%s ${ORIGINALFILE}) - $(tail -n${BADLINESCOUNT} ${ORIGINALFILE} | wc -c)\n" | bc ) ${ORIGINALFILE}

Aileen

Ouch. Way too complicated. – Anemometer

It's long but simplistic in my opinion: modular, human readable, reusable code that can handle massive files and with no obscene bash-isms, sed version requirements, or unavailable packages. – Aileen

Answer 13

To remove the last line in the zshrc file for example:

cat ~/.zshrc
sed -i '' -e '$ d' ~/.zshrc
cat ~/.zshrc

Podium

Answer 14

awk "NR != `wc -l < text.file`" text.file &> newtext.file

This snippet does the trick.

Dahlia

This just erases the whole file for me. – Mejia

Answer 15

You can try this method also : example of removing last n number of lines.

a=0 ; while [ $a -lt 4 ];do sed -i '$ d' output.txt; a=expr $a + 1;done

Removing last 4 lines from file(output.txt).

Gare

Answer 16

Ruby(1.9+)

ruby -ne 'BEGIN{prv=""};print prv ; prv=$_;' file

Planish