How to remove filename prefix with a Posix shell

Question

How can I remove the filename prefix in Bash as in the following example:

XY TD-11212239.pdf

to get

11212239.pdf

i.e, remove XY TD-?

Dunstable

Linux is a kernel. Do you actually mean [bash] or [sh] (or another shell) instead? – Triste

@Shahbaz's edit: While the OP probably meant 'shell', this can actually even be done by writing a C program, for example. What I mean is: Its better to let the OP clarify whether or not he meant 'shell'. – Triste

@ArjunShankar, I removed shell from the title, but by the way he is executing a command (XY file-name) I still believe he wants to do it in a shell. – Nog

@Shahbaz: Hmm.. But the OP says that XY TD- is the 'filename prefix', which he wants to remove. Then 'XY' may not be a shell command. – Triste

@Dunstable - Do you want to rename files removing a prefix? Do you want to do it in a shell, like bash, or in a program (which language?)? – Triste

bash is a superset of POSIX sh. Do you want bash, or do you want POSIX sh? – Fixation

I would say the answer depends on what other files you also want to rename. For example, if you have a bunch of files and for each file name you want to remove all characters up until and including the first dash (-), then try for file in *; do echo mv "$file" "${file#*-}"; done. Of course, remove the echo once the preview looks good. This is analogous to this answer. – Brokerage

Answer 1

You said POSIX shells which would include BASH, Kornshell, Ash, Zsh, and Dash. Fortunately, all of these shells do pattern filtering on variable values.

Patterns are what you use when you specify files with things like * on the Unix/Linux command line:

$ ls *.sh  # Lists all files with a `.sh` suffix

These POSIX shells use four different pattern filtering:

• ${var#pattern} - Removes smallest string from the left side that matches the pattern.
• ${var##pattern} - Removes the largest string from the left side that matches the pattern.
• ${var%pattern} - Removes the smallest string from the right side that matches the pattern.
• ${var%%pattern} - Removes the largest string from the right side that matches the pattern.

Here are a few examples:

foo="foo-bar-foobar"
echo ${foo#*-}   # echoes 'bar-foobar'  (Removes 'foo-' because that matches '*-')
echo ${foo##*-}  # echoes 'foobar' (Removes 'foo-bar-')
echo ${foo%-*}   # echoes 'foo-bar'
echo ${foo%%-*}  # echoes 'foo'

You didn't really explain what you want, and you didn't include any code example, so it's hard to come up with something that will do what you want. However, using pattern filtering, you can probably figure out exactly what you want to do with your file names.

file_name="XY TD-11212239.pdf"
mv "$file_name" "${file_name#*-}" # Removes everything from up to the first dash

Interferometer

Answer 2

You have given very little information, but assuming you are doing this in bash, and have a list of files whose prefix needs to be removed, you can pipe the list through sed:

For example:

./generate_your_list_of_files | sed -e 's/^prefix//'

or if your list of files are space separated:

echo TD-file1 TD-file2.x.yt TD-file3-beep | sed -e 's/\<TD-//g'

The first one matches prefix in the beginning of the line and removes it. The second one matches TD- (or any other prefix you want to substitute) only when it happens at the beginning of a word and replaces it in all the matches in all the lines. This could get dangerous though, for example a file like TD-file\ that\ TD-contains\ space.txt becomes file\ that\ contains\ space.txt

As a side note, don't get your list of files using ls. That's a horrible mistake. If you need to get a list of files to work with and refer to in more than a single place, I'd suggest putting them in an array:

files=(*)

and then work with this array.

---

Due to popular requests (the single comment below), here is how to rename all files in the directory that start with XY TD- such that the prefix is removed (Thanks to @tripleee):

for file in prefix*;
do
    mv "$file" "${file#XY TD-}"
done

Nog

It is also possible that the OP wants to 'rename' the original files. The questions is really too vague to answer sufficiently. – Triste

@tripleee, thanks for letting me know. I am actually only recently learning bash's scripting language. I have updated my answer. – Nog

... And for efficiency and correctness, mv "$file" "${file#XY\ TD-}" with proper quoting and the shell's built-in substitution mechanism. – Seamy

@tripleee, also, thanks for the wonderful link. As a newbie in shell programming, I would have honestly fallen for those useless usages. – Nog

for file in "PM ID-"*; do mv $file echo $file | sed -e 's/^PM ID-//'; done // this code didn't work to remove the prefix PM ID- from file name – Dunstable

@hamid, 'space' is a special character in shell. You need to change PM ID- everywhere to PM\ ID- – Nog

How to change the space for every file as I have sub-folders inside a folder – Dunstable

Your question is starting to get very broad. I suggest asking your question in a separate question. (Hint, the solution involves find. You seem to need to read a bit about bash scripts on your own) – Nog

@Shahbaz, would you consider removing the ill-advised practices from your answer, or moving them to the end to highlight the best-practice approaches suggested by tripleee? It's hard to support this answer as it is when what it puts front-and-center are practices considered harmful. – Fixation

@CharlesDuffy, I avoided the ls. I had written that just to keep things familiar for a noobie and give the warning about it later. But that was really not that important. – Nog

It did not work: files=(*);echo $files|sed -e 's/^new.//'. Why? I use zsh. – Lacewing

Answer 3

You can also use the rename tool

Install it with homebrew

brew install rename

Then run the command below from within the directory to remove the "prefix-" from all files

rename -d prefix- *

Pennypennyaliner

Nice powerfull perl tool! – Psychometrics

Answer 4

In addition to substring extraction shown in other answers, you can also make use of substring replacement. Given the file:

fn="XY TD-11212239.pdf"

Then substring replacement of:

fn_new=${fn//*TD-/}

Will also do what you want.

Kimberlite

Answer 5

$ s='XY TD-11212239.pdf'
$ echo "${s#XY TD-}"
11212239.pdf

If your goal is to perform a mass rename:

while IFS= read -r -d '' filename; do
  mv "$filename" "${filename#XY TD-}"
done < <(find . -type f -name 'XY TD-*' -print0)

Note that <() is a bash extension not present in POSIX sh. You can replace it with a pipeline of the form find ... | while.

Similarly, -print0 is a GNU extension not present in POSIX find. POSIX find provides no equivalent way to locate files which is safe for names containing newlines, making it difficult to replace.

Fixation