What does the double-asterisk (**) wildcard mean?

Question

I've tried the following command but I don't understand the results:

ls **

What does ** mean? How should I use it?

Procrastinate

Answer 1

You're most likely seeing a special feature of some shells that allow wildcard filename patterns to match across directory boundaries, as opposed to a single *, which is a wildcard that matches only within a directory.

If you do not have such a shell, ** will likely be equivalent to *, because "matching zero or more characters followed by zero or more characters" is the same as just "matching zero or more characters".

But if you do have such a shell, ** will match all files and directories in the current directory and subdirectories, whereas * only matches files and directories in the current directory. (In both cases "dot files", those with names starting with ., are not matched).

**'s real power comes when you use it in more specific patterns. For example, you can specify all .txt files no matter what subdirectory they are in with **/*.txt, whereas *.txt only matches those in the current directory.

You should look at the wildcard matching rules for your shell to know for sure what your shell is doing. For example, the bash manual says:

*
Matches any string, including the null string. When the 'globstar' shell option is enabled, and '*' is used in a filename expansion context, two adjacent '*'s used as a single pattern will match all files and zero or more directories and subdirectories. If followed by a '/', two adjacent '*'s will match only directories and subdirectories.

In recent versions of bash the 'globstar' shell option is disabled by default. Enabled via:

shopt -s globstar

I believe zsh also supports this syntax.

It's important to keep in mind that wildcards are expanded by the shell, not by the ls command. If you type ls **, or ls *.txt, the ls command itself never sees the * characters; it only sees an expanded list of files matching the pattern, just as if you had typed the entire list on the command line.

Somatotype

I edited the explanation, smoothing out the flow, adding an example, and nixing the reference to "recursive" which means something different... (as Inigo Montoya would say, "You keep using that word. I do not think it means what you think it means." 😉) – Mcguigan

Yes, ZSH supports this syntax by default. – Adaminah

Answer 2

Globbing

By using the double asterisk (**), you are using a glob to list files on a filesystem. A glob is a string of literal or wildcard characters used for matching the file paths. Using one or more globs for locating files on a filesystem is called globbing.

Apart from Linux shells, globbing is also used in various configuration files to specify the list of files to locate. For example: files and folders to ignore in the .gitignore file, files and include options in tsconfig.json file in Typescript projects etc.

Following are some of the most important aspects of the globbing and double asterisk (**) is one of them:

Segments and Separators (/)

The separator is always the / character. A segment is everything that comes between the two separators.

Example: Tests/HelloWorld.js

Here, Tests and HelloWorld.js are the segments and / is the separator.

Single Asterisk (*)

Single Asterisk (*) matches zero or more characters within one segment. It is used for globbing the files within one directory.

Example: *.js

This glob will match files such as HelloWorld.js but not files like Tests/HelloWorld.js or Tests/UI/HelloWorld.js

Double Asterisk (**)

Double Asterisk (**) matches zero or more characters across multiple segments. It is used for globbing files that are in nested directories.

Example: Tests/**/*.js

Here, the file selecting will be restricted to the Tests directory. The glob will match the files such as Tests/HelloWorld.js, Tests/UI/HelloWorld.js, Tests/UI/Feature1/HelloWorld.js.

Note that the option globstar has to be enabled for ** to have this functionality, and it is disabled by default in Bash on most platforms. To enable it in Bash, run:

shopt -s globstar

Question Mark(?)

Question mark(?) matches a single character within one segment. When some files or directories differ in their name by just one character, you can use the ?.

Example: tests/?at.js

This will match files such as tests/cat.js, test/Cat.js, test/bat.js etc.

Square Brackets ([abc])

Square Brackets ([...]) globs the files with a single character mentioned in the square brackets.

Example: tests/[CB]at.js

This glob will match files like tests/Cat.js or tests/Bat.js

Square Brackets Range ([a-z])

Square Brackets Range ([a-z]), matches one character specified in the range.

Example: tests/feature[1-9]/HelloWorld.js

This glob will match files like tests/feature1/HelloWorld.js, test/feature2/HelloWorld.js and so on... up to 9.

Negation (!)

Negation (!) can be used for excluding some files.

Example 1: tests/[!C]at.js

This will exclude the file tests/Cat.js and will match files like tests/Bat.js, tests/bat.js, tests/cat.js. In other words, [!C] is a negated bracket expression where the character that can be matched is any character except the ones enumerated within [!...]. (As with regular character classes, you can use a dash to enumerate a range, like [!A-Z].)

Negation is also used in configuration files inside an array to negate or exclude some files.

Example 2: ['Tests/**/*.js', '!Tests/UI/**']

This will exclude all files and folders from Tests/UI directory.

However, this variant of negation is not valid in the shell, only the facility to negate a character class.

Deutzia

Note: the globstar shell option is disabled by default in recent bash versions, so the ** (double asterisk) will not match the Tests/HelloWorld.js. To enable it use shopt -s globstar and shopt -u globstar to disable. – Crazy

What about curly braces { }? I thought they can be used with globbing as well. – Amaras

@AdmiralAdama, {...} is an expansion, not a globbing operator. Expansion occurs before globbing. So, for example, *.{js,ts,java} will result in *.js *.ts *.java. – Deutzia

Answer 3

for visual people

The other answers are hard to grok for visual people like me. Here is an illustration fully confirmed by tests. It shows nuances of ** that are not obvious when reading the textual definition.

The directory structure shown below has these properties:

• four directory levels (counting the root)
• two files f at each level with different file extensions
• two directories with the same name o, on different branches at different depths

I tested all the patterns in the table headings below against this structure, using the following command in Bash with globstar enabled: stat -f "%N" <pattern>.

.
├── f.js
├── f.md
└── x
    ├── f.js
    ├── f.md
    ├── o
    │   ├── f.js
    │   ├── f.md
    │   └── z
    │       ├── f.js
    │       └── f.md
    └── y
        ├── f.js
        ├── f.md
        └── o
            ├── f.js
            └── f.md

Comparing * to **

	*	**	*/	**/	*/*.md	**/*.md	*/o/*	**/o/*	**/o/**
f.js	✅	✅
f.md	✅	✅				✅
x	✅	✅	✅	✅
x/f.js		✅
x/f.md		✅			✅	✅
x/o		✅		✅					✅
x/o/f.js		✅					✅	✅	✅
x/o/f.md		✅				✅	✅	✅	✅
x/o/z		✅		✅			✅	✅	✅
x/o/z/f.js		✅							✅
x/o/z/f.md		✅				✅			✅
x/y		✅		✅
x/y/f.js		✅
x/y/f.md		✅				✅
x/y/o		✅		✅					✅
x/y/o/f.js		✅						✅	✅
x/y/o/f.md		✅				✅		✅	✅

Selective deep targeting

Here we selectively target Markdown files in different parts of the directory tree:

	only current dir	anywhere	anywhere under x/o	anywhere under any o	only directly under any o
	*.md	**/*.md	x/o/**/*.md	**/o/**/*.md	**/o/*.md
f.js
f.md	✅	✅
x
x/f.js
x/f.md		✅
x/o
x/o/f.js
x/o/f.md		✅	✅	✅	✅
x/o/z
x/o/z/f.js
x/o/z/f.md		✅	✅	✅
x/y
x/y/f.js
x/y/f.md		✅
x/y/o
x/y/o/f.js
x/y/o/f.md		✅		✅	✅

🚩 **.md is the same as *.md

**.md works like *.md, not like **/*.md. If you append or prepend anything to ** other than /, it will work exactly the same as *.

Mcguigan

what about a folder that has asterisk in it's name? – Duggins

@pablete: * covers also files with an asterisk in its name. If you want to select such files you may use ls *'*'* (just put quotes around the asterisk) – Cemetery

@MaximSuslov or escape it *\** using a backslash. – Revolutionize

it seems that if you simply prepend / to **, it also behaves like *. for example `./**' only shows the files one level deep. – Revolutionize

@Revolutionize I just ran stat -f "%N" ./** and it returned the same results as stat -f "%N" ** (all files, all the way down as show above) + the current directory (because ./** matches ./). – Mcguigan

I have a script that processes all files in a directory. Is there a way to recursively glob all files (and only files)? Currently, I path.is_file() every item to filter out the directories. Based on the chart, I hoped "**/*" or "**/*.*" might work, but they do not. – Discontinuance

@user2514157, It is only a convention, not a rule, that files have extensions and directories do not. Also, there are other file system types besides files and directories. So glob patterns cannot make any assumptions, and only match on the names, not types. You will find an answer to your needs by searching Stack Overflow for "glob only directories files". – Mcguigan

Answer 4

The exact behavior of this particular wildcard has been well covered by the other answers, but information on the general case may be useful.

This behavior is not limited to ls, and is referred to as "globbing", which is the expansion of patterns based on matches with existing filenames. It is important to note that these patterns do not use regular expression syntax.

The shell pre-processes the arguments before they are sent to the program. There are generally multiple levels of expansion, some of these involve globbing.

A great resource for more information on the other wildcards available in a file glob pattern is the unix manpage. A online version for glob can be found here.

Finally, a simple example of what this can do for you, especially when combined with other shell expansion goodies, in this case those provided by the bash shell. Information about the expansions used in this example can be found in the Bash Guide for Beginners - which is my goto resource, despite the title.

ls *{01..04}.{txt,csv} becomes ls *01.txt *01.csv *02.txt *02.csv *03.txt *03.csv *04.txt *04.csv

Which could output something like this:

input_01.txt input_02.txt input_03.txt input_04.txt output_01.csv output_02.csv output_03.csv output_04.csv

While skipping these:

input_05.txt input_06.txt input_07.txt input_08.txt input_09.txt input_10.txt output_05.csv output_06.csv output_07.csv output_08.csv output_09.csv output_10.csv

A trivial example, but if you know that this behavior is not specific to ls, then you can imagine the utility when coupled with mv, cp, rsync, etc.

Countercurrent