Bash script to remove all files and directories except specific ones

Question

I am trying to write a very simple Bash shell script that will cd in a specific directory, it will remove all files and directories except some few selected ones and then cd back to the original dir.

My code is:

#!/bin/bash 
cd /path/to/desired/directory
shopt -s extglob
rm !\(filename1\|filename2\|filename3\) -rf
cd -

I have tried many different ways to write the symbols '(' and '|', with single or double quotes or backslash but nothing worked. Note, that shopt -s extgloband rm !(filename1|filename2) -rf work fine outside a script.

Probably I am committing a standard and fundamental bash-scripting error that I cannot see, but experience is to come...

Any suggestions!? Thanks in advance.

Vicinal

Are you getting any sort of errors? – Gond

Are you sure that rm !(filename1|filename2) -rf works fine outside a script? I'd have expected rm -rf !(filename1|filename2) instead (with the option before the operands). – Rickard

@Rickard I read this several times. I am also using options on rm after the files from time to time. Maybe it's a bit inconsistent but it always worked for me. @mario you don't have to switch back the directory with cd - because a subshell is used when you execute your script. – Rosenwald

@bashphil OK, 'cd -' is a detail, maybe not needed. Thanks for pointing out. – Vicinal

@Rickard Yes I am sure it works. Rhe position of the options -abc... should plays no role. – Vicinal

Answer 1

Two problems I can see right off the bat:

• The -rf argument to rm must come before the filenames
• The extglob specifiers !, (, | and ) should not be escaped with backslashes

Try this instead:

rm -rf !(filename1|filename2|filename3)

If it's still not working, remove the -f argument, and you'll get error messages about what's going wrong instead of silently suppressing them. To print out the name of each file removed, you can add the -v option as well.

Coonskin

just to be little cautious as -rf will remove files without asking and there might be some serious data loss – Necessitate

I found the error. I was confusing bash with sh (whose difference I still don't understand). rm -rf !(filename1|filename2|filename3) as well as rm !(filename1|filename2|filename3) -rf work equally well outside and inside the script. Why the specifiers should not be marked with backslash? – Vicinal

I keep getting bash: !: event not found with bash 4.3.11. It looks like I forgot to add the shopt line. – Hydrodynamic

Answer 2

If you have a recent version of find, then use:

find . -not -regex "filename1|filename2|filename3" -print

Check the output and replace -print with -delete if it prints only the files that you really want to delete.

If your version of find doesn't support -delete, use -type f -exec rm "{}" \;

Note that this will only delete files. This is important: If you say to keep file x but that file is in a folder y, then it would ignore x only to delete it later with the whole folder y.

To get rid of all empty folders (and this preserving the files you want to keep):

find . -type d -empty -exec rmdir "{}" \;

(source: How to Find and Delete Empty Directories and Files in Unix)

Trigg

Answer 3

I don't recommend trying to write a rule to exempt specific files. What I would be more likely to do, would be to try and find either a filemask/glob, or some other anchor in the names of the files that you do want to remove, which the files you want to keep, does not have. In most cases, you could probably do that by extension. Unfortunately, because you haven't mentioned what the names of any of the files are, I can't help you more specifically.

Something like two (or more) find loops:-

find *.mp3 d.mp3 -maxdepth 1 -type f | while read mp3s
do
mv $mp3s ~/d.stuff-to-keep
done    

find *.txt d.mp3 -maxdepth 1 -type f while read textfiles
do
rm -v $textfiles
done

If not, look for some other common characteristic of the files. If there isn't one, you might need to rename your files to give yourself one. I always put lots of anchors (seperated name fields, extensions etc) in my filenames, because that way I have a lot of different ways for doing what I need with them. My usual name format is:-

<extension>.+<author>+<year>+<work-name>+<volume-number>+<number-of-volumes>

Plus signs are field seperators, dashes are for spaces, but with no dashes as either first or last characters. All lowercase letters, and no control characters or other non-alphanumerics allowed. As a result, I can cut those up with awk, cut, or whatever else I might want to use, and rename them easily. Putting the extension at the start of the name, also means that ls only needs one flag (-l) to get a perfectly ordered file system.

If you know how to use your directory structure as a database with the basic POSIX tools, you sometimes won't need to install anything more complicated, in order to do what you want.

You've got a long char limit on filenames in UNIX, as well as tab completion for navigating around long names. Don't be afraid to make liberal use of both of them.

Paltry

Answer 4

Try below:

for i in `ls | grep -v "file1\|file2\|file3"` ; do rm -rf $i; done

Good Luck!

Talkathon

Note that this will fail in all kinds of ways if you have spaces in folder or file names. It also doesn't work with subfolders. – Trigg

@Aaron Digulla as to spaces, try this: for i in ls | grep -v '"new folder"\|file2\|file3' ; do rm -rf $i; done – Talkathon

1. You need quotes around $i, too. 2. "new folder" will be split by for into new and folder. – Trigg

Answer 5

assuming you have files:

drwxr-xr-x 2 root root 4096 Feb 1 02:17 a

drwxr-xr-x 2 root root 4096 Feb 1 02:17 b

drwxr-xr-x 2 root root 4096 Feb 1 02:17 c

drwxr-xr-x 2 root root 4096 Feb 1 02:17 d

drwxr-xr-x 2 root root 4096 Feb 1 02:17 e

drwxr-xr-x 2 root root 4096 Feb 1 02:17 f

drwxr-xr-x 2 root root 4096 Feb 1 02:18 nodelete1

drwxr-xr-x 2 root root 4096 Feb 1 02:18 nodelete2

You could do:

ls | grep -v "nodel.\+" | xargs rm -rf

to use advanced regexp syntax to exclude files.

Explanation:

ls --- lists current directory (ls -d to list only directories)

grep -v --- v flag specifies what NOT TO MATCH + grep allows to write perl style regexp if you supply -P flag

xargs --- applies action "rm -rf" on each of the items

Monarda