Bash problem with eval, variables and quotes

Question

I've been reading about quotes in bash here and everywhere else, but i got no help solving this problem.

The thing is, I have a little script for doing backups in a loop.

If I don't use eval then I have problems with $OPTIONS variable in rsync.

But if I do use eval then the problem goes to the variable $CURRENT_DIR...

rsync returns the following message: 'Unexpected local arg: /path/with'

I've tried every way of quoting the variable $CURRENT_DIR

CURRENT_DIR="/path/with spaces/backup"
DIR="dir_by_project"
f=":/home/project_in_server"
OPTIONS="-avr --exclude 'public_html/cms/cache/**' --exclude 'public_html/cms/components/libraries/cmslib/cache/**' --delete"
eval rsync --delete-excluded -i $OPTIONS  [email protected]$f $CURRENT_DIR/xxx/$DIR/files

Is there any way that i can use the variable $CURRENT_DIR without problems caused by spaces?

Amagasaki

Answer 1

eval rsync --delete-excluded -i $OPTIONS  [email protected]$f "\"$CURRENT_DIR/xxx/$DIR/files\""

command "some thing" executes command with one argument some thing. The quotes are parsed by the shell and arguments are setup as an array when executing the command. The command will see an argument as some thing without the quotes.

The eval command treats its arguments more or less as if they were typed into the shell. So, if you eval command "some thing", bash executes eval with two arguments: command and some thing (again the quotes are eaten while bash sets up the array of arguments). So, eval then acts as if you typed command some thing in the shell, which is not what you want.

What I did was simply to escape the quotes, so that bash passes literally "some thing" including the quotes to eval. eval then acts as if you typed command "some thing".

The outer quotes in my command is not strictly required, they're just habit. You could just as well use:

eval rsync --delete-excluded -i $OPTIONS  [email protected]$f \"$CURRENT_DIR/xxx/$DIR/files\"

Convection

It's working but I don't know exactly why? I guess it has to do with word splitting. Could you please elaborate a little more in your answer? Edit I think I got it now. I used eval because i wanted $OPTIONS to be word splitted and that made CURRENT_DIR to be word splitted too? and when eval runs it remove quotes first... – Amagasaki

Answer 2

Using eval is dangerous, and should be avoided whenever possible. In this case, the primary problem is that you're trying to define OPTIONS as containing multiple words, and bash variables don't handle this very well. There is a solution: put OPTIONS in an array, instead of a simple variable (and use double-quotes around all variable references, to keep spaces from getting treated as word separators).

CURRENT_DIR="/path/with spaces/backup"
DIR="dir_by_project"
f=":/home/project_in_server"
OPTIONS=(-avr --exclude 'public_html/cms/cache/**' --exclude 'public_html/cms/components/libraries/cmslib/cache/**' --delete)
rsync --delete-excluded -i "${OPTIONS[@]}"  "[email protected]$f" "$CURRENT_DIR/xxx/$DIR/files"

Laodicea

+1 for the clear explanation. I only wonder, how portable is this solution. I tried in OS X 10.5 and it worked, but will it work in others OS's??? What will be the minimal bash version required for this to work? – Amagasaki

@Cesar: According to this, linear arrays (the kind I'm using here) were added in bash v2. (Associative arrays are new in bash v4, but they're not needed here.) – Laodicea

Answer 3

BASH FAQ entry #50: "I'm trying to put a command in a variable, but the complex cases always fail!"

Authorized

Underrated comment. Wisdom is worth more than technique. That was a fantastic site. I was trying to make a shell variable overly complicated instead of just having another function that can be overridden. – Kassel

Answer 4

Generic reusable solution

While understanding how to correctly quote things is important, for ease of use and to prevent slip-ups I prefer to use a function:

The following keeps spaces in arguments by quoting each element of array:

function token_quote {
  local quoted=()
  for token; do
    quoted+=( "$(printf '%q' "$token")" )
  done
  printf '%s\n' "${quoted[*]}"
}

Example usage:

$ token_quote token 'single token' token
token single\ token token

Above, note the single token's space is quoted as \.

$ set $(token_quote token 'single token' token)
$ eval printf '%s\\n' "$@"
token
single token
token
$

This shows that the tokens are indeed kept separate.

---

Given some untrusted user input:

% input="Trying to hack you; date"

Construct a command to eval:

% cmd=(echo "User gave:" "$input")

Eval it, with seemingly correct quoting:

% eval "$(echo "${cmd[@]}")"
User gave: Trying to hack you
Thu Sep 27 20:41:31 +07 2018

Note you were hacked. date was executed rather than being printed literally.

Instead with token_quote():

% eval "$(token_quote "${cmd[@]}")"
User gave: Trying to hack you; date
%

eval isn't evil - it's just misunderstood :)

Anadromous

Answer 5

I agree with Gordon

You have no need for eval in this case (you are not forming a variable name from variables, or otherwise making an expression on the fly)

And and you want to double-quote all variable referensces that COULD have spaces that you would want to preserve

But Another good habit is to always refrence variables with {} ...

  "${CURRENT_DIR}" 

instead of

  $CURRENT_DIR

This removes any name ambiguity

Bleareyed

Answer 6

I know probably you have used it already, but, what about single quotes? (this type ' ' ) ?

Virilism

Answer 7

You need to escape space in CURRENT_DIR="/path/with\ spaces/backup" if this doesn't work then put double backslash CURRENT_DIR="/path/with\\ spaces/backup"

Hydrocarbon

this might work around this particular problem but is not a good solution in general ... the value of the variable as defined in the question is correct and reasonable. – Decennial