Stripping single and double quotes in a string using bash / standard Linux commands only

Question

I'm looking for something that will translate a string as follows, using only bash / standard Linux commands:

1. Single-quotes surrounding a string should be removed
2. Double-quotes surrounding a string should be removed
3. Unquoted strings should remain the same
4. Strings with unmatched surrounding quotes should remain the same
5. Single-quotes that don't surround the string should remain
6. Double-quotes that don't surround the string should remain

For example:

• 'Food' should become Food
• "Food" should become Food
• Food should remain the same
• 'Food" should remain the same
• "Food' should remain the same
• 'Fo'od' should become Fo'od
• "Fo'od" should become Fo'od
• Fo'od should remain the same
• 'Fo"od' should become Fo"od
• "Fo"od" should become Fo"od
• Fo"od should remain the same

Thank you!

Robbins

+1 for all of those acceptance tests! – Alleen

As most "customers" do, an extra one's been added afterwards that wasn't in the original set :-) – Benilda

So.. backslash escapes are meaningless? "Food\" -> Food\ ? – Forficate

Answer 1

This should do it:

sed "s/^\([\"']\)\(.*\)\1\$/\2/g" in.txt

Where in.txt is:

"Fo'od'
'Food'
"Food"
"Fo"od'
Food
'Food"
"Food'
'Fo'od'
"Fo'od"
Fo'od
'Fo"od'
"Fo"od"
Fo"od

And expected.txt is:

"Fo'od'
Food
Food
"Fo"od'
Food
'Food"
"Food'
Fo'od
Fo'od
Fo'od
Fo"od
Fo"od
Fo"od

You can check they match with:

diff -s <(sed "s/^\([\"']\)\(.*\)\1\$/\2/g" in.txt) expected.txt

Benilda

Very close :-) echo \"fo\"bar\' | sed "s/([\"'])(.*)\1/\2/g" Got: fobar' Expected: "fo"bar' – Robbins

Great! Just go changing the requirements after the coding is done ;-) updated for that case. – Benilda

Here's one final requirements change :-) echo \"fo\'bar\' | sed -e "s/([\"'])(.*)\1\$/\2/g" Got: "fobar Expected: "fo'bar' – Robbins

I hate regular expressions now. – Benilda

+1 for an entertaining contest. :) Deleting my proven-to-be-dodgy sed post accordingly. – Alleen

+1 for proving that 'Some people, when confronted with a problem, think “I know, I'll use regular expressions.” Now they have two problems.' Is as true as it gets! – Fine

This is a good regex - but the backslash infront of the $ is unnecessary. It could just be sed "s/^([\"'])(.*)\1$/\2/g" in.txt – Roadway

@Roadway you're right. I always escape $ inside "" as in most cases it will get misinterpreted as a variable. This is one case where it does not (but escaping it doesn't harm either). – Benilda

good point @Benilda when doing $ inside quotes it will be interpreted as a variable. Although for all cases when writing regex in sed and matching EOL char, you should generally be safe because the next character will be a "/". Either way, no harm done and its safe. Good call – Roadway

Answer 2

You could use tr:

echo "$string" | tr -d 'chars to delete' 

... also works, however 'tr' is known to be problematic on much older (circa Redhat 9-ish) distributions. tr is an abbreviation for 'translate', commonly used in pipes to transform input. The -d option simply means 'delete'.

Most modern versions also contain predefined macros to transform upper to lower, lower to upper, kill white space, etc. Hence, if you use it, take a second to poke at what else it does (see the help output / man page), comes in handy.

Marchetti

Answer 3

VAR="'FOOD'"

VAR=$(eval echo $VAR)

Explanation: Since quotes are already understood by the shell you can ask the shell to evaluate a command that just echos the quoted string, the same way it does when you type it yourself.

Here, eval echo $VAR expands to eval echo 'FOOD' because the quotes are actually part of the value of VAR. If you were to run echo 'FOOD' into the shell you'd get FOOD (without the quotes). That's what eval does: it takes its input and runs it like a shell command.

⚠CODE INJECTION!

eval expose scripts to code injection.

VAR=';ls -l'

VAR=$(eval echo $VAR)

will cause execution of ls -l.

Much more harmful codes could be injected here.

Salacious

please add some explanation too!! – Sellingplater

I added some explanation and modernized the example a bit. – Yorgos

Answer 4

You probably want to use sed...

echo $mystring | sed -s "s/^\(\(\"\(.*\)\"\)\|\('\(.*\)'\)\)\$/\\3\\5/g"

Naturalism

I have not idea if it works, but +1 for the awesomeness of your sed command :) – Quintero

This is very close, but just a teeny bit off. echo \"fo\"bar\' | sed -s "s/^(\"(.*)\")\|(\'(.*)\')\$/\\2/g" Got: fobar' Expected: "fo"bar' – Robbins

Yes... for some reason I can't fathom, the last $ stubbornly refuses to match the end of my string. Working on it... – Naturalism

Also, quote your parameter expansion. – Forficate

Answer 5

Just using Bash builtins (i.e. Bash parameter expansion):

IFS=' ' 

food_strings=( "'Food'" '"Food"' Food "'Food\"" "\"Food'" "'Fo'od'" "\"Fo'od\"" "Fo'od" "'Fo\"od'" '"Fo"od"' 'Fo"od'  )  

for food in ${food_strings[@]}; do 

   [[ "${food#\'}" != "$food" ]] && [[ "${food%\'}" != "$food" ]] && { food="${food#\'}"; food="${food%\'}"; } 

   [[ "${food#\"}" != "$food" ]] && [[ "${food%\"}" != "$food" ]] && { food="${food#\"}"; food="${food%\"}"; } 

   echo "$food"

done 

For yet another example of Bash parameter expansion see:

http://codesnippets.joyent.com/posts/show/1816

Kristankriste

Answer 6

Just stumbled upon this as well. For the first three test cases, eval echo $string works well. To get it to work for all cases requested and a few others, I came up with this (tested with bash and dash):

#!/bin/sh

stripquotes() {
    local firstchar="`substr "$1" 0 1`"
    local len=${#1}
    local ilast=$((${#1} - 1))
    local lastchar="`substr "$1" $(($len - 1))`"
    if [ "$firstchar" = '"' ] || [ "$firstchar" = "'" ] && [ $firstchar = $lastchar ]; then
        echo "`substr "$1" 1 $(($len - 2))`"
    else
        echo "$1"
    fi
}

# $1 = String.
# $2 = Start index.
# $3 = Length (optional). If unspecified or an empty string, the length of the
#      rest of the string is used.
substr() {
    local "len=$3"
    [ "$len" = '' ] && len=${#1}
    if ! (echo ${1:$2:$len}) 2>/dev/null; then
        echo "$1" | awk "{ print(substr(\$0, $(($2 + 1)), $len)) }"
    fi
}

var="'Food'"
stripquotes "$var"

var='"Food"'
stripquotes "$var"

var=Food
stripquotes "$var"

var=\'Food\"
stripquotes "$var"

var=\"Food\'
stripquotes "$var"

var="'Fo'od'"
stripquotes "$var"

var="\"Fo'od\""
stripquotes "$var"

var="Fo'od"
stripquotes "$var"

var="'Fo\"od'"
stripquotes "$var"

var="\"Fo\"od\""
stripquotes "$var"

var="Fo\"od"
stripquotes "$var"

# A string with whitespace should work too.
var="'F\"o 'o o o' o\"d'"
stripquotes "$var"

# Strings that start and end with the same character that isn't a quote or
# doublequote should stay the same.
var="TEST"
stripquotes "$var"

# An empty string should not cause errors.
var=
stripquotes "$var"

# Strings of length 2 that begin and end with a quote or doublequote should not
# cause errors.
var="''"
stripquotes "$var"
var='""'
stripquotes "$var"

Ensilage

Answer 7

python -c "import sys;a=sys.stdin.read();a=a.strip();print (a[1:-1] if a[0]==a[-1] and a[0] in \"'\\\"\" else a)"

it doesn't handle edge cases extremely well (such as an empty string), but it will serve as a starting point. It works by striping the front and back character if they are the same and if they are ' or "

Irina

Jason explicitly said that he wants bash / standard Linux commands. Even though python is shipped with most of the commonly used distros, it is certainly not standard. If you install a basic Debian system, for example, it doesn't include python. – Fine