Most compact URL encoding of JSON data?

Question

Some time ago I saw a website that encoded game replay data into unicode characters so that you could copy-and-paste the address to share it with friends. I basically want to do the same thing -- I've got some data in a JavaScript object and I want to put it into the address bar.

I know I can use jQuery.param or something like that, but I want a much more compact format.

Turns out this is a little tricky to Google. Most search results turn up URL shorteners, which is distinctly different. I am not trying to store a short URL in my database for later lookup. All the data should be in the URL itself.

What encoding can I use for this? A link or description of the algorithm is fine, but impementations for Node/JavaScript would be even better.

I only need to support the latest version of Chrome stable, so I don't care if older browsers can't handle the unicode. I think I'll put the data in the fragment portion (after #) so I only need to support whatever encoding rules apply to that piece. I think just about anything is allowed, no?

Droughty

Answer 1

There you go https://github.com/qntm/base65536 & https://www.npmjs.com/package/base65536

Base65536 is a binary encoding optimised for UTF-32-encoded text and Twitter. This JavaScript module, base65536, is the first implementation of this encoding.

It featured on hackernews some time ago.

Cymoid

Aha! That solves half the problem. The other half is how do I serialize my JS object into a buffer? JSON.stringify? msgpack? – Droughty

Yeah I think so, you need a regular string so you are able to encode, then you decode and JSON.parse it. – Cymoid

BTW "For transmitting data through Twitter, Base65536 is now considered obsolete; see Base2048." – Leavetaking

Answer 2

Here's an implementation of enapupe's suggestion:

import Msgpack from 'msgpack-lite';
import Base65536 from 'base65536';

export function uniencode(obj) {
    return Base65536.encode(Msgpack.encode(obj));
}

export function unidecode(str) {
    return Msgpack.decode(Base65536.decode(str));
}

Usage:

let o = {foo:'bar',baz:[1,2,3,4,5],quux:'corge'};
console.log(uniencode(o).length,JSON.stringify(o).length); // 24, 46

i.e. uniencode is about half the length of JSON.stringify, or even less according to Twitter (.length doesn't count unicode properly. Twitter counts the above as 15 chars). A better measurement is:

function len(str) {
    return [...str].length;
}

Which gives 15 and 46 respectively.

Droughty

Why do I get this error : base65536__WEBPACK_IMPORTED_MODULE_9___default.a.encode is not a function – Hunch

@Hunch You've got a problem with your webpack setup my dude. Maybe open a new question? – Droughty

Fixed in [email protected] github.com/qntm/base65536/issues/21 – Hunch