URL component encoding in Node.js

Question

I want to send http request using node.js. I do:

http = require('http');

var options = {
    host: 'www.mainsms.ru',
    path: '/api/mainsms/message/send?project='+project+'&sender='+sender+'&message='+message+'&recipients='+from+'&sign='+sign
    };

    http.get(options, function(res) {
    console.log('STATUS: ' + res.statusCode);
    console.log('HEADERS: ' + JSON.stringify(res.headers));
    }).on('error', function(e) {
    console.log('ERROR: ' + e.message);
    });

When my path like this:

/api/mainsms/message/send?project=geoMessage&sender=gis&message=tester_response&recipients=79089145***&sign=2c4135e0f84d2c535846db17b1cec3c6

Its work. But when message parameter contains any spaces for example tester response all broke. And in console i see that http use this url:

  /api/mainsms/message/send?project=geoMessage&sender=gis&message=tester

How to send spaces. Or i just can't use spaces in url?

Wrench

You need to do URL encoding. – Qulllon

@thefourtheeye: Can you give any examples or links to read? – Wrench

developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/… and nodejs.org/api/querystring.html – Qulllon

Possible duplicate of How do I URl encode something in Node.js? – Selenaselenate

Answer 1

What you are looking for is called URL component encoding.

path: '/api/mainsms/message/send?project=' + project + 
'&sender=' + sender + 
'&message=' + message +
'&recipients=' + from + 
'&sign=' + sign

has to be changed to

path: '/api/mainsms/message/send?project=' + encodeURIComponent(project) +
'&sender=' + encodeURIComponent(sender) +
'&message=' + encodeURIComponent(message) + 
'&recipients='+encodeURIComponent(from) +
'&sign=' + encodeURIComponent(sign)

Note:

There are two functions available. encodeURI and encodeURIComponent. You need to use encodeURI when you have to encode the entire URL and encodeURIComponent when the query string parameters have to be encoded, like in this case. Please read this answer for extensive explanation.

Qulllon

Necromancer here ... This answer is not valid, encodeURIComponent is a window property, so it's only available in browsers, not Node.js (see title). – Adjudge

@JulienCROUZET Actually encodeURIComponent is available in Node.js as well. It is provided by v8. Please check this out. – Qulllon

It /may/ be available, it is defined in ECMA but has no existance in Node.js doc, so no LTS or other assurances for it. The /Official/ way is the querystring module – Adjudge

I am not sure I understand. Are you saying I should not suggest functions like Array, String etc just because they are not in Node.js documentation? – Qulllon

To clarify thefourtheye’s point, it seems odd to consider node’s homegrown querystring module (which has been effectively deprecated by URLSearchParams anyway) to be more "official" than functionality which has been specified by a formal international standard as part of the language itself for nearly two decades :) – Feeder

As stupid as it may look, since it's not part of the Node.js documentation, encodeURIComponent is the right answer. Use it like it is without any requires/imports – Stonewall

Answer 2

The question is for Node.js. encodeURIcomponent is not defined in Node.js. Use the querystring.escape() method instead.

var qs = require('querystring');
qs.escape(stringToBeEscaped);

Jairia

Why is this not the accepted answer? I believe it should be. – Lingenfelter

When the question was originally posted (2013), encodeURIComponent was not available in Node.js. Even in 2016, it was not available in many packaged versions of Node.js, even though it was available in, say, version 6. Question of timing I guess? – Jairia

Answer 3

The best way is to use the native module QueryString :

var qs = require('querystring');
console.log(qs.escape('Hello $ é " \' & ='));
// 'Hello%20%24%20%C3%A9%20%22%20\'%20%26%20%3D'

This is a native module, so you don't have to npm install anything.

Adjudge

Answer 4

TLDR: Use fixedEncodeURI() and fixedEncodeURIComponent()

MDN encodeURI() Documentation...

function fixedEncodeURI(str) {
   return encodeURI(str).replace(/%5B/g, '[').replace(/%5D/g, ']');
}

MDN encodeURIComponent() Documentation...

function fixedEncodeURIComponent(str) {
 return encodeURIComponent(str).replace(/[!'()*]/g, function(c) {
   return '%' + c.charCodeAt(0).toString(16);
 });
}

Why to Not Use encodeURI() and encodeURIComponent()

It is really not recommended to use encodeURI() and encodeURIComponent(), as they are insufficient to properly handle URI or URL encoding directly. Just like at this piece...

'&message=' + encodeURIComponent(message) + 

Let's say that the message var is set to this: "Hello, world! (Really hello!)". So what is that evaluated to?

console.log('&message=' + encodeURIComponent("Hello, world! (Really hello!)"));

Output:

&message=Hello%2C%20world!%20(Really%20hello!)

That's not right! Why weren't ( and ) encoded? After all, according to RFC3986, Section 2.2: Reserved Characters...

If data for a URI component would conflict with a reserved character's purpose as a delimiter, then the conflicting data must be percent-encoded before the URI is formed.

sub-delims = "!" / "$" / "&" / "'" / "(" / ")" / "*" / "+" / "," / ";" / "="

Parens are a sub-delim, but they are not being escaped by encodeURI() or encodeURIComponent().

Motion