Shell variable with spaces , quoting for single command line option

Question

Autoconf scripts have trouble with a filename or pathname with spaces. For example,

./configure CPPFLAGS="-I\"/path with space\""

results in (config.log):

configure:3012: gcc  -I"/path with space"  conftest.c  >&5
gcc: with: No such file or directory
gcc: space": No such file or directory

The compile command from ./configure is ac_compile='$CC -c $CFLAGS $CPPFLAGS conftest.$ac_ext >&5' and I am not able to modify this (I could perhaps, but working around autoconf in this way is not a general solution).

I think it comes down to getting a shell variable that contains spaces to be parsed as a single command line variable rather than split at spaces. The simplest shell example I can come up with is to create a file with spaces and attempt to list is with ls with a shell variable as the argument to ls:

$ touch "a b"
$ file="a b"
$ ls $file
ls: a: No such file or directory
ls: b: No such file or directory

This works, but is illegal since in autoconf I can't modify the shell code:

$ ls "$file"
a b

None of the following attempts at quoting things work:

$ file="\"a \"b"; ls $file
ls: "a: No such file or directory
ls: b": No such file or directory
$ file="a\ b"
$ file="a\\ b"
$ file="`echo \\"a b\\"`"

and so on.

Is this impossible to accomplish in shell scripts? Is there a magical quoting that will expand a shell variable with spaces into a single command line argument?

Shawn

Answer 1

You should try to set the $IFS environment variable.

from man bash(1):

IFS - The Internal Field Separator that is used for word splitting after expansion and to split lines into words with the read builtin command. The default value is ''space tab newline''.

For example

IFS=<C-v C-m>  # newline
file="a b"
touch $file
ls $file

Don't forget to set $IFS back or strange things will happen.

Llamas

I would set IFS to a null string; it works just as well. saveIFS="$IFS"; IFS=''; ls $file; IFS="$saveIFS" – Nefarious

Answer 2

if you give command

 gcc -I"x y z"

in a shell then certainly the single command line parameter "-Ix y z" will be passed to gcc. There is no question to that. That's the whole meaning of double quotes: things inside double quotes are NOT subject to field splitting, and so not subject to $IFS either, for instance.

But you need to be careful about the number of quotes you need. For instance, if you say

 file="a b" # 1

and then you say

 ls $file # 2

what happens is that the file variable's contents are 'a b', not '"a b"', because the double quotes were "eaten" when line 1 was parsed. The replaced value is then field-separated and you get ls on two files 'a' and 'b'. The correct way to get what you want is

 file="a b"; ls "$file"

Now the problem in your original case is that when you set a variable to a string that CONTAINS double quotes, the double quotes are later not interpreted as shell quote symbols but just as normal letters. Which is why when you do something like

 file="\"a b\""; ls $file

actually the shell tokenizes the contents of the file variable into '"a' and 'b"' when the ls command is analyzed; the double quote is no longer a shell quote character but just part of the variable's contents. It's analogous to that if you set

 file="\$HOME"; ls $file

you get an error that '$HOME' directory does not exist---no environment variable lookup takes place.

So your best options are

1. Hack autoconf
2. Do not use path names with spaces (best solution)

Polyptych

Answer 3

Using space in directory names in the Unix world is simply asking for trouble. It's not just the problem of quoting in shell scripts (which needs to be done right anyway): some tools simply cannot cope with spaces in filenames. For instance, you can't (portably) write a Makefile rule that says build baz.o from foo bar/baz.c.

In the case of CPPFLAGS above, I would try one of the following (in order of preference):

1. Fix the system not use use any space in directory names.

2. Write a small wrapper around the compiler and call ./configure CC=mygcc. In that case mygcc might be:

   #!/bin/sh
   gcc "-I/foo bar/include" "$@"
   

3. Create a symbolic link (e.g., /tmp/mypath) to the dreaded path and use CPPFLAGS=-I/tmp/mypath.

Varied

Answer 4

You want to quote the entire argument, in either of these ways:

./configure "CPPFLAGS=-I/path with space"
./configure CPPFLAGS="-I/path with space"

The ./configure command then sees a single argument

"CPPFLAGS=-I/path with space"

which is parsed as a parameter named«CPPFLAGS» having the value«-I/path with space» (brackets added for clarity).

Exult

Answer 5

Using quotes is interesting. From (lightly) reading the bash man page I thought you had to escape the space with \, thus "/path with space" becomes /path\ with\ space I've never tried the quotes, but it seems that it doesn't work generally (your ls example). Escaping works with ls without quoting and without changing IFS.

What happens if you use the "escaping spaces" format of the command?

Tootle

The point is to use ls as a test and not touch its argument. The requirement is to be able to do ls $file exactly like that and only make changes to file="a b". This is an analog to changing the argument to configure without modifying the files involved. – Nefarious

Answer 6

$ file="\"a b\""

$ eval ls $file

Coriolanus

This won't solve the problem because the autoconf script will need modification to call eval. The point of the question is to avoid doing that modification. – Hunch

Answer 7

Everything depends on how the variable is used. First, note that if you are using Autoconf, this probably means that make will be used eventually, so that the rules are dictated by make, and in particular, the default make rules. Even though you may want to use your own rules exclusively, things must remain consistent between tools, and some variables have standard meanings, so that you do not want to deviate from them. This is not the case of CPPFLAGS, but this should remain similar to CFLAGS, which is standard. See the POSIX make utility, where variables are simply expanded with standard sh word splitting, which does not provide any quoting mechanism (the field separator is controlled by $IFS, but do not change the IFS variable to accept spaces as normal characters since this will break other things, like being able to provide several -I and/or -L options in such variables with the standard way).

Since there is such a limitation with make, I suppose that it would be useless to try to avoid this limitation in Autoconf.

Now, since a space is necessarily a field separator, the only possibility is to provide pathnames without space characters. If spaces in pathnames were to be supported in the future, this would probably be done via pathname encoding, with decoding at the high-level UI (a bit like with URL's). Alternatively, if you have the choice and really want to use spaces in pathnames, you may use some non-ASCII space (BTW, this is how RISC OS supports space in pathnames, by forcing it to be the no-break space).

Bonkers