Add a prefix string to beginning of each line

Question

I have a file as below:

line1
line2
line3

And I want to get:

prefixline1
prefixline2
prefixline3

I could write a Ruby script, but it is better if I do not need to.

prefix will contain /. It is a path, /opt/workdir/ for example.

Saylor

Answer 1

# If you want to edit the file in-place
sed -i -e 's/^/prefix/' file

# If you want to create a new file
sed -e 's/^/prefix/' file > file.new

If prefix contains /, you can use any other character not in prefix, or escape the /, so the sed command becomes

's#^#/opt/workdir#'
# or
's/^/\/opt\/workdir/'

Hesson

if prefix contains / then its more easy to use awk. – Calutron

@benjamin, I had already upvoted your answer, however, I prefer sed for lightweight tasks such as this. If "prefix" is known, it's very easy to pick a character not from "prefix". – Hesson

Don't forget you can also use sed in a pipeline, e.g. foo | sed -e 's/^/x /' | bar. – Bursa

@AlokSinghal: How can one apply your answer to all the lines in the file except the first line (header)? – Selectee

@AlokSinghal: I think I found out my answer: sed -e '1!s/^/prefix/' file. – Selectee

@Selectee cool. Another way would be sed -e '2,$s/^/prefix/'. – Hesson

Maybe also mention that if you have the prefix in a variable, you can't use single quotes. sed "s%^%$PREFIX%" file (using double quotes to enable variable interpolation; and % as an alternative delimiter, on the assumption that $PREFIX might contain slashes, but won't contain percent signs). – Intolerant

@BinChen escape the / like \/ (in single-quoted strings) or \\/ (in double-quoted strings) – Transform

Use sed -e 's/$/postfix/' file if you want to add string to the end of each line. – Trumaine

If you process large inputs, consider giving perl a shot. perl -ne $| = 1;print "prefix$_". In my test this was ~80 times faster than sed. For more see this this blogpost rc3.org/2014/08/28/surprisingly-perl-outperforms-sed-and-awk by twitter.com/rafeco – Brittbritta

for adding tab space after the word sed -i -e 's/^/prefix\t/' testfile.txt – Cannelloni

You don't need to use / as the delimiter with sed, you can use something like s.^./. to add a slash at the beginning of each line. You can use any character as a delimiter. – Transform

@Dev - yes, I mentioned that in my answer. – Hesson

No, you didn't. You mentioned that you can escape /. – Transform

I wrote: "If prefix contains /, you can use any other character not in prefix", followed by an example without the /. – Hesson

Why do you use the -e option and could you also explain why you don't put the g at the end? – Meantime

I use -e out of habit, because sometimes I want more than one command in the same sed invocation. In this case, g at the end won't really do anything, since the pattern can only match once per line. – Hesson

The example with sed as listed appears to skip lines that have only a new-line on them (empty lines). Awk as listed elsewhere on this page handles all the lines. – Extravascular

@Extravascular Weird. I just tried it with a test file, and it doesn't skip empty lines. – Hesson

could it be added only the prefix does not exist at the beginning? – Somme

@Somme this probably should be a separate question but the following works for me: sed '/^prefix/!s/^/prefix/g' file – Hesson

Answer 2

awk '$0="prefix"$0' file > new_file

In awk the default action is '{print $0}' (i.e. print the whole line), so the above is equivalent to:

awk '{print "prefix"$0}' file > new_file

With Perl (in place replacement):

perl -pi 's/^/prefix/' file

Calutron

With a pipe/stream or variable: prtinf "$VARIABLE\n" | awk '$0="prefix"$0' – Location

With a large file (12 G), awk reports awk: out of memory in readrec 1 source line number 1, but the solution with sed completes successfully. – Fluoroscope

This is the best answer, with AWK it Worked right off the bat without having to annoyingly deal with escaping regex special characters – Cosset

With "normal files" split into lines awk shouldn't run out of memory... – Globular

Answer 3

You can use Vim in Ex mode:

ex -sc '%s/^/prefix/|x' file

1. % select all lines

2. s replace

3. x save and close

Candent

For me, I just open the file in vim and type :%s/^/prefix/, since this strategy ends up being useful in many situations – Revenge

Answer 4

If your prefix is a bit complicated, just put it in a variable:

prefix=path/to/file/

Then, you pass that variable and let awk deal with it:

awk -v prefix="$prefix" '{print prefix $0}' input_file.txt

Solano

Answer 5

Here is a oneliner solution using the ts command from moreutils

$ cat file | ts prefix

And how it's derived step by step:

# Step 1. create the file

$ cat file
line1
line2
line3

# Step 2. add prefix to the beginning of each line

$ cat file | ts prefix
prefix line1
prefix line2
prefix line3

Note that the prefix will be space separated from the content

Aesthetically

'ts' is not installed by default on many Linux distros. Also, downvoting because the trailing "tr -d ' '" in this answer will remove all spaces from the lines, not just the space that was added by 'ts' – Airwaves

Answer 6

If you have Perl:

perl -pe 's/^/PREFIX/' input.file

Modification

Answer 7

Using & (the whole part of the input that was matched by the pattern”):

cat in.txt | sed -e "s/.*/prefix&/" > out.txt

OR using back references:

cat in.txt | sed -e "s/\(.*\)/prefix\1/" > out.txt

Piefer

Answer 8

Using the shell:

#!/bin/bash
prefix="something"
file="file"
while read -r line
do
 echo "${prefix}$line"
done <$file > newfile
mv newfile $file

Zoo

Answer 9

While I don't think pierr had this concern, I needed a solution that would not delay output from the live "tail" of a file, since I wanted to monitor several alert logs simultaneously, prefixing each line with the name of its respective log.

Unfortunately, sed, cut, etc. introduced too much buffering and kept me from seeing the most current lines. Steven Penny's suggestion to use the -s option of nl was intriguing, and testing proved that it did not introduce the unwanted buffering that concerned me.

There were a couple of problems with using nl, though, related to the desire to strip out the unwanted line numbers (even if you don't care about the aesthetics of it, there may be cases where using the extra columns would be undesirable). First, using "cut" to strip out the numbers re-introduces the buffering problem, so it wrecks the solution. Second, using "-w1" doesn't help, since this does NOT restrict the line number to a single column - it just gets wider as more digits are needed.

It isn't pretty if you want to capture this elsewhere, but since that's exactly what I didn't need to do (everything was being written to log files already, I just wanted to watch several at once in real time), the best way to lose the line numbers and have only my prefix was to start the -s string with a carriage return (CR or ^M or Ctrl-M). So for example:

#!/bin/ksh

# Monitor the widget, framas, and dweezil
# log files until the operator hits <enter>
# to end monitoring.

PGRP=$$

for LOGFILE in widget framas dweezil
do
(
    tail -f $LOGFILE 2>&1 |
    nl -s"^M${LOGFILE}>  "
) &
sleep 1
done

read KILLEM

kill -- -${PGRP}

Frecklefaced

use the -u option to sed to avoid the buffering. – Foofaraw

Buffering can be turned off with unbuffer/stdbuf, see unix.stackexchange.com/q/25372/6205 – Sprayberry

Answer 10

Using ed:

ed infile <<'EOE'
,s/^/prefix/
wq
EOE

This substitutes, for each line (,), the beginning of the line (^) with prefix. wq saves and exits.

If the replacement string contains a slash, we can use a different delimiter for s instead:

ed infile <<'EOE'
,s#^#/opt/workdir/#
wq
EOE

I've quoted the here-doc delimiter EOE ("end of ed") to prevent parameter expansion. In this example, it would work unquoted as well, but it's good practice to prevent surprises if you ever have a $ in your ed script.

Atheist

Answer 11

Here's a wrapped up example using the sed approach from this answer:

$ cat /path/to/some/file | prefix_lines "WOW: "

WOW: some text
WOW: another line
WOW: more text

prefix_lines

function show_help()
{
  IT=$(CAT <<EOF
    Usage: PREFIX {FILE}

    e.g.

    cat /path/to/file | prefix_lines "WOW: "

      WOW: some text
      WOW: another line
      WOW: more text
  )
  echo "$IT"
  exit
}

# Require a prefix
if [ -z "$1" ]
then
  show_help
fi

# Check if input is from stdin or a file
FILE=$2
if [ -z "$2" ]
then
  # If no stdin exists
  if [ -t 0 ]; then
    show_help
  fi
  FILE=/dev/stdin
fi

# Now prefix the output
PREFIX=$1
sed -e "s/^/$PREFIX/" $FILE

Warfare

This will not work if PREFIX contains any characters special to sed like a slash. – Datum

Good point... If you find you use slash alot, you could use a different delimiter with the sed part, as detailed here, which would allow you to use it in searches. Other special sed chars can be put in by escaping with a slash, e.g prefix_lines \* – Warfare

Answer 12

1. You can also achieve this using the backreference technique

   sed -i.bak 's/\(.*\)/prefix\1/' foo.txt
   

2. You can also use with awk like this

   awk '{print "prefix"$0}' foo.txt > tmp && mv tmp foo.txt
   

Lobito

Answer 13

Using Pythonize (pz):

pz '"preix"+s' <filename

Cuyler

Answer 14

You can do it using AWK

echo example| awk '{print "prefix"$0}'

or

awk '{print "prefix"$0}' file.txt > output.txt

For suffix: awk '{print $0"suffix"}'

For prefix and suffix: awk '{print "prefix"$0"suffix"}'

Mcmillian

Answer 15

If you need to prepend a text at the beginning of each line that has a certain string, try following. In the following example, I am adding # at the beginning of each line that has the word "rock" in it.

sed -i -e 's/^.*rock.*/#&/' file_name

Lineament

Answer 16

Simple solution using a for loop on the command line with bash:

for i in $(cat yourfile.txt); do echo "prefix$i"; done

Save the output to a file:

for i in $(cat yourfile.txt); do echo "prefix$i"; done > yourfilewithprefixes.txt

Unapt

Answer 17

For people on BSD/OSX systems there's utility called lam, short for laminate. lam -s prefix file will do what you want. I use it in pipelines, eg:

find -type f -exec lam -s "{}: " "{}" \; | fzf

...which will find all files, exec lam on each of them, giving each file a prefix of its own filename. (And pump the output to fzf for searching.)

Notepaper

You're right, it appears this is a BSD only command. POSIX replaced it with paste, but paste doesn't have the feature of adding a full separator string. I'll update my answer. – Notepaper

Answer 18

SETLOCAL ENABLEDELAYEDEXPANSION

YourPrefix=blabla

YourPath=C:\path

for /f "tokens=*" %%a in (!YourPath!\longfile.csv)     do (echo !YourPrefix!%%a) >> !YourPath!\Archive\output.csv

Blacken

Please add some explanation to your answer such that others can learn from it – Legnica

Please don't post only code as an answer, but also provide an explanation of what your code does and how it solves the problem of the question. Answers with an explanation are usually more helpful and of better quality, and are more likely to attract upvotes. – Lucky

If you were looking for an answer in a knowledgebase, trying to solve an issue, do you think this answer would be helpful? Not so much. – Wrand

Thank you for your interest in contributing to the Stack Overflow community. This question already has quite a few answers—including one that has been extensively validated by the community. Are you certain your approach hasn’t been given previously? If so, it would be useful to explain how your approach is different, under what circumstances your approach might be preferred, and/or why you think the previous answers aren’t sufficient. Can you kindly edit your answer to offer an explanation? – Toreutics