When to wrap quotes around a shell variable?

Asked 8/4, 2012 at 23:2 Answered 16/1, 2018 at 17:14

321

Should or should I not wrap quotes around variables in a shell script?

For example, is the following correct:

xdg-open $URL
[ $? -eq 2 ]

xdg-open "$URL"
[ "$?" -eq "2" ]

And if so, why?

Lately answered 8/4, 2012 at 23:2 Comment(10)

See also unix.stackexchange.com/questions/171346/… – Bdellium 23/12, 2015 at 9:32

This question gets a lot of duplicates, many of which are not about variables, so I retitled to "value" instead of "variable". I hope this helps more people find this topic. – Bdellium 28/3, 2017 at 9:47

@Trinetta What's up with the reverted edit? – Bdellium 12/4, 2017 at 6:15

See also I just assigned a variable, but echo $variable shows something else – Bdellium 6/6, 2018 at 11:20

Related: Difference between single and double quotes in Bash as well. – Trinetta 13/9, 2018 at 21:54

See mywiki.wooledge.org/Quotes – Gatt 19/4, 2019 at 0:16

Bash is a hack that ended up being used well beyond what its designs considered. There are better ways to do things but there is no "correct / secure way". I say this because there are a lot of references here that will all have opposing opinions and it can become very confusing especially for people that are used to the newer languages and tools designed for specific tasks. – Kamilah 21/4, 2019 at 4:5

@Heavy Gray: What do you suggest instead? PowerShell? – Catullus 17/10, 2021 at 20:49

Perhaps see also #4412738 which covers complications related to quoting an ssh command line. In very brief, you need to escape some shell metacharacters from both the local and the remote shell. Using a here document if you can might alleviate the problems somewhat. – Bdellium 15/1, 2022 at 12:21

See also https://mcmap.net/q/12295/-which-characters-need-to-be-escaped-when-using-bash – Saccharate 22/4, 2022 at 17:20

258

General rule: quote it if it can either be empty or contain spaces (or any whitespace really) or special characters (wildcards). Not quoting strings with spaces often leads to the shell breaking apart a single argument into many.

$? doesn't need quotes since it's a numeric value. Whether $URL needs it depends on what you allow in there and whether you still want an argument if it's empty.

I tend to always quote strings just out of habit since it's safer that way.

Photographic answered 8/4, 2012 at 23:8 Comment(14)

so you would alternate between quoting and non-quoting variables in your scripts? thanks for your response – Lately 8/4, 2012 at 23:10

I think it's worth adding to this answer what the effect of not quoting a variable with spaces would be. – Hearth 8/4, 2012 at 23:13

do you only have to quote string variables? – Lately 8/4, 2012 at 23:16

@Cristian: All shell variables are string variables (well, except thing like arrays). When a shell variable is set to an integer, it's still a string, it's just that the string happens to be a sequence of digits. – Baguio 9/4, 2012 at 0:52

@GordonDavisson thanks for the response, does that mean the programmer has to know in advance what kind of variable it will be, like if it will be a single character or number or boolean, and add quotes appropriately? or should you just add quotes to every variable? thanks for the reply! – Lately 9/4, 2012 at 10:50

Note that "spaces" really means "any whitespace". – Burial 9/4, 2012 at 15:13

@Cristian: If you aren't sure what might be in the variable, it's safer to quote it. I tend to follow the same principle as paxdiablo, and just make a habit of quoting everything (unless there's a specific reason not to). – Baguio 9/4, 2012 at 17:1

If you don't know the value of IFS, quote it no matter what. If IFS=0, then echo $? can be very surprising. – Ibbison 8/2, 2017 at 4:4

Important: You also need to quote if the variable's value may contain wildcards such as *. – Overmeasure 12/6, 2017 at 21:12

Quote based on the context, not on what you expect the values to be, otherwise your bugs will be worse. For example, you are sure that none of your paths have spaces, so you think you can write cp $source1 $source2 $dest, but if for some unexpected reason dest doesn't get set, the third argument just disappears, and it will silently copy source1 over source2 instead of giving you an appropriate error for the blank destination (as it would have if you had quoted each argument). – Zasuwa 24/1, 2018 at 16:22

Derek, I believe that would be adequately covered by my 'contain spaces or be blank' comment. – Photographic 24/1, 2018 at 20:49

@Photographic "I tend to always quote strings just out of habit since it's safer that way."; agree, my first tought was then: "always quote Bash variable" – Sech 20/10, 2018 at 19:36

in addition, there are two cases where quotes are not necessary variable assignmnent a=$b and tests between [[ ... ]], in the later the double quotes can change the meaning for example a=*, [[ x = $a ]] succeeds whereas [[ x = "$a" ]] fails – Stellastellar 31/10, 2018 at 9:34

quote it if... has the thought process backwards - quotes aren't something you add when you need to, they're something you remove when you need to. Always wrap strings and scripts in single quotes unless you need to use double quotes (e.g. to let a variable expand) or need to use no quotes (e.g. to do globbing and file name expansion). – Gatt 19/4, 2019 at 0:6

173

In short, quote everything where you do not require the shell to perform word splitting and wildcard expansion.

Single quotes protect the text between them verbatim. It is the proper tool when you need to ensure that the shell does not touch the string at all. Typically, it is the quoting mechanism of choice when you do not require variable interpolation.

$ echo 'Nothing \t in here $will change'
Nothing \t in here $will change

$ grep -F '@&$*!!' file /dev/null
file:I can't get this @&$*!! quoting right.

Double quotes are suitable when variable interpolation is required. With suitable adaptations, it is also a good workaround when you need single quotes in the string. (There is no straightforward way to escape a single quote between single quotes, because there is no escape mechanism inside single quotes -- if there was, they would not quote completely verbatim.)

$ echo "There is no place like '$HOME'"
There is no place like '/home/me'

No quotes are suitable when you specifically require the shell to perform word splitting and/or wildcard expansion.

Word splitting (aka token splitting);

 $ words="foo bar baz"
 $ for word in $words; do
 >   echo "$word"
 > done
 foo
 bar
 baz

By contrast:

 $ for word in "$words"; do echo "$word"; done
 foo bar baz

(The loop only runs once, over the single, quoted string.)

 $ for word in '$words'; do echo "$word"; done
 $words

(The loop only runs once, over the literal single-quoted string.)

Wildcard expansion:

$ pattern='file*.txt'
$ ls $pattern
file1.txt      file_other.txt

By contrast:

$ ls "$pattern"
ls: cannot access file*.txt: No such file or directory

(There is no file named literally file*.txt.)

$ ls '$pattern'
ls: cannot access $pattern: No such file or directory

(There is no file named $pattern, either!)

In more concrete terms, anything containing a filename should usually be quoted (because filenames can contain whitespace and other shell metacharacters). Anything containing a URL should usually be quoted (because many URLs contain shell metacharacters like ? and &). Anything containing a regex should usually be quoted (ditto ditto). Anything containing significant whitespace other than single spaces between non-whitespace characters needs to be quoted (because otherwise, the shell will munge the whitespace into, effectively, single spaces, and trim any leading or trailing whitespace).

When you know that a variable can only contain a value which contains no shell metacharacters, quoting is optional. Thus, an unquoted $? is basically fine, because this variable can only ever contain a single number. However, "$?" is also correct, and recommended for general consistency and correctness (though this is my personal recommendation, not a widely recognized policy).

Values which are not variables basically follow the same rules, though you could then also escape any metacharacters instead of quoting them. For a common example, a URL with a & in it will be parsed by the shell as a background command unless the metacharacter is escaped or quoted:

$ wget http://example.com/q&uack
[1] wget http://example.com/q
-bash: uack: command not found

(Of course, this also happens if the URL is in an unquoted variable.) For a static string, single quotes make the most sense, although any form of quoting or escaping works here.

wget 'http://example.com/q&uack'  # Single quotes preferred for a static string
wget "http://example.com/q&uack"  # Double quotes work here, too (no $ or ` in the value)
wget http://example.com/q\&uack   # Backslash escape
wget http://example.com/q'&'uack  # Only the metacharacter really needs quoting

The last example also suggests another useful concept, which I like to call "seesaw quoting". If you need to mix single and double quotes, you can use them adjacent to each other. For example, the following quoted strings

'$HOME '
"isn't"
' where `<3'
"' is."

can be pasted together back to back, forming a single long string after tokenization and quote removal.

$ echo '$HOME '"isn't"' where `<3'"' is."
$HOME isn't where `<3' is.

This isn't awfully legible, but it's a common technique and thus good to know.

As an aside, scripts should usually not use ls for anything. To expand a wildcard, just ... use it.

$ printf '%s\n' $pattern   # not ``ls -1 $pattern''
file1.txt
file_other.txt

$ for file in $pattern; do  # definitely, definitely not ``for file in $(ls $pattern)''
>  printf 'Found file: %s\n' "$file"
> done
Found file: file1.txt
Found file: file_other.txt

(The loop is completely superfluous in the latter example; printf specifically works fine with multiple arguments. stat too. But looping over a wildcard match is a common problem, and frequently done incorrectly.)

A variable containing a list of tokens to loop over or a wildcard to expand is less frequently seen, so we sometimes abbreviate to "quote everything unless you know precisely what you are doing".

Bdellium answered 30/12, 2014 at 7:58 Comment(6)

This is a variant of (part of) an answer I posted to a related question. I am pasting it here because this is succinct and well-defined enough to become a canonical question for this particular problem. – Bdellium 30/12, 2014 at 7:59

I will note that this is item #0 and a recurring theme on the mywiki.wooledge.org/BashPitfalls collection of common Bash mistakes. Many, many of the individual items on that list are basically about this issue. – Bdellium 28/1, 2017 at 12:39

Token splitting is called word splitting in the Bash reference. See my edit. gnu.org/software/bash/manual/html_node/Word-Splitting.html – Forwardlooking 9/11, 2021 at 12:16

@Forwardlooking Thanks! I reworded the text to prefer the official term. – Bdellium 9/11, 2021 at 12:21

null bytes should get a honorable mention as shit that will change even when they shouldn't, for example: NULL="$(php -r 'echo chr(0);')" + printf "%s" "$NULL" | wc -c will print 0... – Wacker 17/5, 2022 at 21:13

@Wacker I don't feel that's a suitable topic to include here, though it can indeed be a pesky problem. The shell and various utilities use C strings, which reserve the null byte as string terminator. Some tools like Perl and Python (and some versions of Awk) use a more sophisticated string representation internally, and are able to accommodate completely arbitrary strings. – Bdellium 18/5, 2022 at 4:18

Here is a three-point formula for quotes in general:

Double quotes

In contexts where we want to suppress word splitting and globbing. Also in contexts where we want the literal to be treated as a string, not a regex.

Single quotes

In string literals where we want to suppress interpolation and special treatment of backslashes. In other words, situations where using double quotes would be inappropriate.

No quotes

In contexts where we are absolutely sure that there are no word splitting or globbing issues or we do want word splitting and globbing.

Examples

Double quotes

literal strings with whitespace ("StackOverflow rocks!", "Steve's Apple")
variable expansions ("$var", "${arr[@]}")
command substitutions ("$(ls)", "`ls`")
globs where directory path or file name part includes spaces ("/my dir/"*)
to protect single quotes ("single'quote'delimited'string")
Bash parameter expansion ("${filename##*/}")

Single quotes

command names and arguments that have whitespace in them
literal strings that need interpolation to be suppressed ( 'Really costs $$!', 'just a backslash followed by a t: \t')
to protect double quotes ('The "crux"')
regex literals that need interpolation to be suppressed
use shell quoting for literals involving special characters ($'\n\t')
use shell quoting where we need to protect several single and double quotes ($'{"table": "users", "where": "first_name"=\'Steve\'}')

No quotes

around standard numeric variables ($$, $?, $# etc.)
in arithmetic contexts like ((count++)), "${arr[idx]}", "${string:start:length}"
inside [[ ]] expression which is free from word splitting and globbing issues (this is a matter of style and opinions can vary widely)
where we want word splitting (for word in $words)
where we want globbing (for txtfile in *.txt; do ...)
where we want ~ to be interpreted as $HOME (~/"some dir" but not "~/some dir")

Hot tags

Godot Unity Godot Help Programming Godot 4.X GUI GDScript 3D 2D Physics CSharp Godot 3.X VR XR Projects C++

Recommended topics

Hot tags