Standardized way to serialize JSON to query string?

Question

I'm trying to build a restful API and I'm struggling on how to serialize JSON data to a HTTP query string.

There are a number of mandatory and optional arguments that need to be passed in the request, e.g (represented as a JSON object below):

{
   "-columns" : [
      "name",
      "column"
   ],
   "-where" : {
      "-or" : {
         "customer_id" : 1,
         "services" : "schedule"
      }
   },
   "-limit" : 5,
   "return" : "table"
}

I need to support a various number of different clients so I'm looking for a standardized way to convert this json object to a query string. Is there one, and how does it look?

Another alternative is to allow users to just pass along the json object in a message body, but I read that I should avoid it (HTTP GET with request body).

Any thoughts?

Edit for clarification:

Listing how some different languages encodes the given json object above:

• jQuery using $.param: -columns[]=name&-columns[]=column&-where[-or][customer_id]=1&-where[-or][services]=schedule&-limit=5&return=column
• PHP using http_build_query: -columns[0]=name&-columns[1]=column&-where[-or][customer_id]=1&-where[-or][services]=schedule&-limit=5&return=column
• Perl using URI::query_form: -columns=name&-columns=column&-where=HASH(0x59d6eb8)&-limit=5&return=column
• Perl using complex_to_query: -columns:0=name&-columns:1=column&-limit=5&-where.-or.customer_id=1&-where.-or.services=schedule&return=column

jQuery and PHP is very similar. Perl using complex_to_query is also pretty similar to them. But none look exactly the same.

Merola

why not just using POST requests? – Gemstone

check if this helps, api.jquery.com/jQuery.param – Fortepiano

@akonsu: Because I'm trying to fetch (GET) some data, not POST new data. – Merola

Are you looking to do this on the server side, and if so, what language? If on the client side, can you use jquery? – Nonappearance

@Fortepiano - That's jQuery specific, right? Is it standardized? – Merola

@Andreas, ok, if you do not want to use POST then why not url-encoding your JSON and putting it in to a single query string parameter? – Gemstone

@JoeFrambach: The API is on the server side, written in PERL 5, but that does not matter I believe. I do not control in which languages the clients will be written in. – Merola

@akonsu: Yes exactly! But the encoding seems to be done different in different languages, hence the question if there is a standardized way. – Merola

I always thought that url encoding is language independent. you just replace some characters with their hexadecimal representation preceeded by a % character. en.wikipedia.org/wiki/Percent-encoding – Gemstone

Honestly I dont know if there is a standardized way of doing that, I think its all about the language that you are using. for jquery you can use the one i sent you. you can use other libraries in dotnet. Or you can create your own function to serialize and deserialize JSON to querystring and back. – Fortepiano

Edited the post with examples from jQuery, PHP and PERL. – Merola

@akonsu: Seems like every language has its own implementation. – Merola

yes, but the result is the same across all these implementations. you have HTTP protocol, and every language has its own clients. but the protocol is still the same. same here. – Gemstone

@Jubair: If I would to my own serialization/deserialization, how would clients, written in different languages, use them? – Merola

@akonsu: I posted results from some different languages, and they are not the same :/ – Merola

possible duplicate of Serialize JSON to query string in JavaScript/jQuery – Dudeen

Answer 1

URL-encode (https://en.wikipedia.org/wiki/Percent-encoding) your JSON text and put it into a single query string parameter. for example, if you want to pass {"val": 1}:

mysite.com/path?json=%7B%22val%22%3A%201%7D

Note that if your JSON gets too long then you will run into a URL length limitation problem. In which case I would use POST with a body (yes, I know, sending a POST when you want to fetch something is not "pure" and does not fit well into the REST paradigm, but neither is your domain specific JSON-based query language).

Gemstone

Yea that is probably one way to accomplish this. I doubt that I will run into the URL length limitation problem. I wonder what's best: 1) Pass a single json parameter 2) Use POST (although we're GETting) or 3) Use GET with a message body. Maybe support both 1) and 3) ? – Merola

By the way, the "JSON-based query langague" I'm using comes from SQL::Abstract::More: search.cpan.org/~dami/SQL-Abstract-More-1.11/lib/SQL/Abstract/… – Merola

I would implement 1) and 2). I would use 1) as much as I can and when url gets too long I would use POST. A GET with a body I do not like because it is weird, although it is technically possible. – Gemstone

Note there is no "official" URL length limit. In theory URLs can have any length, but due to restrictions of some implementations (e. g. old proxy servers) only URLs up to 2048 are considered "safe". – Cloudless

Answer 2

There is no single standard for JSON to query string serialization, so I made a comparison of some JSON serializers and the results are as follows:

JSON:    {"_id":"5973782bdb9a930533b05cb2","isActive":true,"balance":"$1,446.35","age":32,"name":"Logan Keller","email":"[email protected]","phone":"[phone]","friends":[{"id":0,"name":"Colon Salazar"},{"id":1,"name":"French Mcneil"},{"id":2,"name":"Carol Martin"}],"favoriteFruit":"banana"}
Rison:   (_id:'5973782bdb9a930533b05cb2',age:32,balance:'$1,446.35',email:'[email protected]',favoriteFruit:banana,friends:!((id:0,name:'Colon Salazar'),(id:1,name:'French Mcneil'),(id:2,name:'Carol Martin')),isActive:!t,name:'Logan Keller',phone:'[phone]')
O-Rison: _id:'5973782bdb9a930533b05cb2',age:32,balance:'$1,446.35',email:'[email protected]',favoriteFruit:banana,friends:!((id:0,name:'Colon Salazar'),(id:1,name:'French Mcneil'),(id:2,name:'Carol Martin')),isActive:!t,name:'Logan Keller',phone:'[phone]'
JSURL:   ~(_id~'5973782bdb9a930533b05cb2~isActive~true~balance~'!1*2c446.35~age~32~name~'Logan*20Keller~email~'logankeller*40artiq.com~phone~'*2b1*20*28952*29*20533-2258~friends~(~(id~0~name~'Colon*20Salazar)~(id~1~name~'French*20Mcneil)~(id~2~name~'Carol*20Martin))~favoriteFruit~'banana)
QS:      _id=5973782bdb9a930533b05cb2&isActive=true&balance=$1,446.35&age=32&name=Logan Keller&[email protected]&phone=[phone]&friends[0][id]=0&friends[0][name]=Colon Salazar&friends[1][id]=1&friends[1][name]=French Mcneil&friends[2][id]=2&friends[2][name]=Carol Martin&favoriteFruit=banana
URLON:   $_id=5973782bdb9a930533b05cb2&isActive:true&balance=$1,446.35&age:32&name=Logan%20Keller&[email protected]&phone=+1%20(952)%20533-2258&friends@$id:0&name=Colon%20Salazar;&$id:1&name=French%20Mcneil;&$id:2&name=Carol%20Martin;;&favoriteFruit=banana
QS-JSON: isActive=true&balance=%241%2C446.35&age=32&name=Logan+Keller&email=logankeller%40artiq.com&phone=%2B1+(952)+533-2258&friends(0).id=0&friends(0).name=Colon+Salazar&friends(1).id=1&friends(1).name=French+Mcneil&friends(2).id=2&friends(2).name=Carol+Martin&favoriteFruit=banana

The shortest among them is URL Object Notation.

Asthenic

JSON is a standard (see ECMA-404)! You have to keep the order and quote all strings with double quotes. (like the JSON: ... line you have.) – Canadian

@AlexisWilke the libraries convert the JSON into a format suitable for URL's and other usecases. You then use the library to decode the URL back to real JSON. var url = urlon.encode(JSON) var JSON = urlon.decode(url) – Ululate

Answer 3

How about you try this sending them as follows:

http://example.com/api/wtf?
[-columns][]=name&
[-columns][]=column&
[-where][-or][customer_id]=1&
[-where][-or][services]=schedule&
[-limit]=5&
[return]=table&

I tried with a REST Client

And on the server side (Ruby with Sinatra) I checked the params, it gives me exactly what you want. :-)

Sharonsharona

How standard is this? If you open your API as public would you need to document your ad-hoc serialization or is there some RFC you can reference? – Accede

Answer 4

Another option might be node-querystring. It also uses a similar scheme to the ones you've so far listed.

It's available in both npm and bower, which is why I have been using it.

Dudeen

Answer 5

Works well for nested objects.

Passing complex objects as query parameters of a url. In the example below, obj is the JSON object to pass into query parameters.

Injecting JSON object as query parameters:

value = JSON.stringify(obj);

URLSearchParams to convert a string to an object representing search params. toString to retain string type for appending to url:

queryParams = new URLSearchParams(value).toString();

Pass the query parameters using template literals:

url = `https://some-url.com?key=${queryParams}`;

Now url will contain the JSON object as query parameters under key (user-defined name)

Extracing JSON from url:

This is assuming you have access to the url (either as string or URL object)

url_obj = new URL(url); (only if url is NOT a URL object, otherwise ignore this step)

Extract all query parameters in the url:

queryParams = new URLSearchParams(url_obj.search);

Use the key to extract the specific value:

obj = JSON.parse(queryParams.get('key').slice(0, -1));

slice() is used to extract a tailing = in the query params which is not required.

Here obj will be the same object passed in the query params.

I recommend to try these steps in the web console to understand better.

You can test with JSON examples here: https://json.org/example.html

Selfappointed