Possible to urlencode a variable in a shell script?

Question

Is it possible to url encode a variable within a shell script?

#!/bin/bash 

now=$(date +"%T") 

DATA=$(wget -q -O -   "http://someurl.com/x.htm?callback=webRequest&exthrs=1&extMode=&fund=1&entitlement=0&skipcache=&extendedMask=1&partnerId=2&output=json&noform=1")

wget -q -O - "http://somewhere.com?abc=$1&responseData=$DATA"

echo "-----COMPLETE----- $now   $1 $RANDOM  
"

I want to url encode the DATA variable, since its results have & in it, it messes up the params in the second wget, is there a way to url encode that DATA variable without using PHP to url encode?

Mordant

Answer 1

Here is one method for URL encoding the shell string DATA:

DATA=$(python -c "import urllib, sys; print urllib.quote(sys.argv[1])"  "$DATA")

Here is another:

DATA=$(perl -MURI::Escape -e 'print uri_escape($ARGV[0]);' "$DATA")

Erigena

Yes perfect, they both work (y) thanks. I thought maybe there was a builtin way in shell to do it, but it's okay. Out of perl/python which one has less spend on CPU/Mem you think? Many instances of this script are to be ran concurrently, so every bit helps. – Mordant

@SvenKahn On my system (Debian stable), the perl version is faster. – Erigena

Thank you, I appreciate that you looked into it. – Mordant

I tried this but it converts = signs which messes up some receiving servers. – Parapsychology

@Parapsychology - It should include equal signs because that's part of the specification (w3schools.com/tags/ref_urlencode.asp?_sm_au_=iVVDMg0TSmrMV6Dm). You are not supposed to encode the entire query string (the "?a=b&c=d" string), you only encode the names and values (the "a", "b", "c" and "d") – not the question mark, equal signs, or the ampersands. – Gao

Answer 2

By the way, the way to do this in Python 3 is this:

DATA=$(python3 -c "import urllib.parse, sys; print(urllib.parse.quote(sys.argv[1]))" "$DATA")

Expanse