Including base64 gzipped stylesheets/images in javascript?

Question

I know you can include css and images, among other file types, which have been stored in base64 form within a javascript file. However, those are decently huge... and gzipped, they shrink down a LOT, even with the ~33% overhead from base64 encoding.

Non-gzipped, images are data:image/gif;base64, data:image/jpeg, data:image/png, and css is data:text/css;base64. What mime type can/should I be using, then, to include css or image data URIs which are gzipped? (Or if gzip+base64 can't work, is there any other compression I can do to bring down the string's size, while still keeping the data stored within the javascript?)

..edit.. I think the question is being misunderstood. I am not asking if I should include gzipped base64 strings within javascript. Yes, I know it's best, in most cases, to gzip the javascript and other files on the server end. But that is not applicable for a userscript; a userscript has no server, and consists of only a single file. Firefox allows a @require directive, but Opera and Chrome do not, and local file security issues come into play with loading any local files. Thus anything needed by the script has to be either: 1) on the web (slow) or 2) embedded in the userscript (big).

Now this question assumes that big is preferable to slow, but that big does not have to mean we totally ignore just how big; if it can be smaller, that's an improvement.

So assuming that a base64 string is embedded in javascript, the question is how to make it into something meaningful.

Either:

1) atob() can convert raw base64-encoded gzip to raw gzip within javascript. (atob does not need to know the mediatype). The question then would be how to decompress that raw gzipped css or image file so that the resulting output can be fed into the document.

or 2) given the proper mediatype, browsers at least theoretically (per the datauri RFC) should be able to load any file directly from a datauri. "" is sufficient to load a non-gzipped css stylesheet. The question here would be what link type attribute and datauri mediatype combination should work (and which browsers would it work for)? Preferably, for a userscript, this would be a combination that works in Opera, FF, and Chrome.

Nearby

the base64 encoding has 33% overhead. – Recognition

Fixed. :) I don't really care about the overhead's size; for a userscript, size isn't as important to me as keeping everything to one single distributable file. But it'd be nice to keep the base64 blobs as small as I can, hence this Q. – Nearby

Wouldn't you want to just include the base64 in your JS, and gzip the JS? – Lethargic

if size is not that important for you, why do you want to reduce vhe base64 blob size: is it for development readability? – Lota

Well, that, and also just sheer js file bloat. It still has to be parsed in, right? So if I have several icons, and a stylesheet or two, that can easily be 50 or 60k of base64. gzipped, then base64, it's significantly less: (real world example) 9917 bytes of base64 from text source, vs 3306 bytes of base64 from gzipped text source. – Nearby

Oh yes, and several browsers have a max size for base64 in a js file; somewhere in the neighborhood of 64k. So this would expand that limit. – Nearby

Is this a general question about handling compressed data in JavaScript or a specific question about compressed data in data URLs? – Adactylous

Whichever can yield a functional solution: within javascript, either a) how a compressed string within a datauri can be decompressed, or b) how a raw gzip string (or other compression type) might be decompressed, either one to a string usable by js (for HTML insertion, etc.) – Nearby

@Nearby As already said, data URLs with compressed data of any type is not practical as none of the browsers would support it. So you would need to implement and invoke the decoding process on your own. And at that point it doesn’t matter what type of data format (data URL or custom format) you use as you would be the only one that can handle it. – Adactylous

@Gumbo: I've not seen yet in my tests any mediatype/etc settings which have support by any browser for decoding a compressed datauri, so I'd tend to agree. Theoretically, they should support it, but in practice, none seem to have implemented it. So that then would take us to b) above - some way in which a raw binary compressed string (gzip or otherwise) can be decompressed and read as text by javascript, preferably without the js decompression code itself being massive. :D – Nearby

@Nearby I think this is not supported because the server is able to gzip the complete source and it does not make sense to let the browser unzip multiple resources instead of one and I think it results a smaller file if you gzip multiple data-uri's and the html source in one package. – Tallowy

@mgutt, that doesn't seem to have anything to do with userscripts. The userscript is in a different context (and typically from a different author) than the html. Gzipping 1+ data uris and the html source together is not possible, nor is a userscript provided by the server anyhow. – Nearby

@Lethargic Wouldn't you want to just include the base64 in your JS, and gzip the JS? I'd just include it uri encoded (not base64) as the gzip will be way more efficient afterward + no b64 overhead. – Hire

Answer 1

In HTTP, compression is most often only applied for transmission to reduce the payload that is to be transmitted. This is done by the Content-Encoding header field.

But the data URL scheme is very limited and you can only specify the media type:

dataurl    := "data:" [ mediatype ] [ ";base64" ] "," data

Although you could use a multipart message, most user agents don’t support them in data URLs. It would also be questionable whether the additional data to describe such a multipart message wouldn’t be more than the data you safe by compressing the actual payload.

So compressing the data in a data URL is possible in theory but impracticable. It is better to simply compress the whole document the data URL is embedded in.

Adactylous

This doesn't address the question, nor is it correct. It's fully legal, per the dataurl RFC, to encode just about anything. Nor does gzip require a multipart mime type; application/gzip is fully legal, for example. Nor, for that matter, is base64 the only permitted encoding - base64 is not even the default encoding (US-ASCII is). – Nearby

Of course you can wrap any data in a data URL by specifying the corresponding media type. And you can also do it with application/gzip data. But then the data is handled as application/gzip data and not as CSS or JavaScript or whatever. And the Base64 encoding is also only used for wrapping to have a URL-safe charset the data needs to be encoded according to the URL-encoding. – Adactylous

Sure, but again, this does not address the question of how to include a gzipped css file. I haven't seen a complete implementation, but I've heard of it being done, somehow, via an XMLHttpRequest with overriden mime type, sending the gzipped string through a dummy XMLHttpRequest not so the data goes anywhere, but so that the browser then decompresses it into the responseText. The pseudofunction I've seen was: – Nearby

function read(fileName) { var xmlhttp = new XMLHttpRequest(); xmlhttp.open("GET", fileName, false); xmlhttp.overrideMimeType('text/plain; charset=x-user-defined'); xmlhttp.send(null); return xmlhttp.responseText; } – Nearby

So you send compressed data to the server to decompress it? What would that be good for? – Adactylous

The idea, as I saw it, was to let the browser decompress it on the return, as the browser has built-in gzip decompression, while javascript doesn't. – Nearby

Then what would be the benefit of compressing it in the first place anyway? You use compression to reduce transmission the payload. Sending the compressed data back to the server to receive it again would make the advantage of having less data to transmit obsolete. – Adactylous

As stated, this is a userscript. That is, it is transmitted once, not repeatedly. The point of compression is to reduce the size of the script for the sake of reducing the size of the script, not reducing the amount retransmitted. Also, if you look at that pseudocode, they're not sending it back to the server, they're trying to do a local loopback, thus there is no retransmission to/from the server in any case. – Nearby