How to compress a string using GZip or similar in Dart?

Question

I want to compress a string in Dart (in the browser). I tried this:

import 'package:archive/archive.dart';

[...]

List<int> stringBytes = UTF8.encode(myString);
List<int> gzipBytes = new GZipEncoder().encode(stringBytes);
String compressedString = UTF8.decode(gzipBytes, allowMalformed: true);

Obviously UTF8.decode is not intended for this and it doesn't work (file is unreadable).

What is the right way to compress a string in Dart?

Lorraine

What are you trying to achieve? I mean what are you trying to do with compressed strings? Can you post more of your code, including writing to a file? – Soulless

pub.dartlang.org/packages/archive – Luff

@GünterZöchbauer Yes, that's what I am using – Lorraine

Sorry, missed the import. I got the impression you tried it with plain Dart. – Luff

@ArgentiApparatus I encode the string as a data URI and download it, as described here https://mcmap.net/q/75489/-using-html5-javascript-to-generate-and-save-a-file Works perfectly, but I'd like to compress it first. Compressing it manually as gzip decreases size a lot. – Lorraine

What is the uncompressed data in the file and what do you intend to do with it? – Soulless

I have a feeling that you are attempting to save the raw compressed bytes as a file by downloading it from the browser. I think maybe what you actually want to do is create a text file containing your string then put that in a zip archive file, and download that. – Soulless

@ArgentiApparatus I'm making a webapp. The string is XML that contains the user's work. It's saved with a custom file extension and can later be opened again. – Lorraine

Answer 1

The compressed list of bytes is probably not a valid UTF8 sequence instead you could encode it in base64.

import 'dart:convert';
import 'package:archive/archive.dart';

void main() {
  var myString = 'myString';
  var stringBytes = utf8.encode(myString);
  var gzipBytes = GZipEncoder().encode(stringBytes);
  var compressedString = base64.encode(gzipBytes);

  print(compressedString);
}

Freidafreight

Or just create a download file from the raw compressed bytes – Soulless

@ArgentiApparatus Dart's Uri.encodeComponent method, which you need to prepare a file for downloading inside a browser, takes a string and not a list of bytes. – Lorraine

I see. So if the method you are using to dynamically create a download file must deal with a string, you can BASE64 encode the data bytes as in the above answer, note that it will increase the size of your download somewhat though. – Soulless

@ArgentiApparatus Yep, I've noticed that. For one of my test files: Original size: 727 kB, Compressed manually: 236 kB, Compressed with BASE64 encoding: 331 kB. Still a great reduction. – Lorraine

Just discovered I can use BASE64URL.encode instead so I can skip Uri.encodeComponent. – Lorraine

how to decompress it back? – Korean

@maheshmnj new GZipDecoder().decodeBytes(BASE64.decode(compressedString)) – Lorraine

Answer 2

You can use this function below if you want.

import 'dart:convert';
import 'dart:io';

  void _compress(String json) {
    final enCodedJson = utf8.encode(json);
    final gZipJson = gzip.encode(enCodedJson);
    final base64Json = base64.encode(gZipJson);

    final decodeBase64Json = base64.decode(base64Json);
    final decodegZipJson = gzip.decode(decodeBase64Json);
    final originalJson = utf8.decode(decodegZipJson);
    }

Cherokee

This answer is up-to-date, no need to import a third party library. – Cowbell

Why isn't this the best answer? even though this answer is correct and doesn't require any third party library – Quilmes

Heads up: this won't likely work for the web because this relies on 'dart:io' which doesn't support the web. – Extrasystole

this should not be the best answer as gzip isn't supported by Flutter Web as of 2023 @Quilmes – Biforked

Answer 3

String data=' ';
for(int i=0;i<10;i++){
    data=data+'Hello World!\r\n';
}
var original=utf8.encode(data);
var compressed=gzip.encode(original);
var decompressed=gzip.decode(compressed);

Dressmaker

Thank you for contributing an answer. Would you kindly edit your answer to to include an explanation of your code? That will help future readers better understand what is going on, and especially those members of the community who are new to the language and struggling to understand the concepts. – Guadalajara