Android Image upload/download with Base64 into JSON causes Out of memory error

Question

I currently encode and decode images to Base64. I overcame the initial issue with OOM's with the use of streams to encode the images into strings.

My issue now is that I cannot fathom how to add multiple Base64 encoded strings for multiple resolutions images (5620 x 3747 - 4.92MB or 3264 x 1836 - 1.35MB) to a JSON Object via Gson. Currently Gson throws an OOM exception only with 2 Base64 Strings from a 5312 x 2988 - 4.95 MB Image.

I understand that android may only be able to spare 16/20Mb per application, so this conversion must be way over the limit.

How can I write the Base64 String in a stream to a JSON object that will contain the specific values needed to post into my server?

Would it be easier to change my server to accept a Multi-Part request instead of a JSON based POJO with multiple Base64 Strings? I currently use Volley and there isn't an official Multi-Part Request as well as IO streaming.

If it's a matter of compression, how much compression should I apply to the image before encoding into a Base64 String? I ideally want to lose barely any quality but have optimal compression levels.

Bit more Information

I am uploading multiple different resolution images as it is a test for compatibility. For example, all the images that I am sending up have been taken on low resolution and extremely high resolution devices as my App relies on these images for functionality. I am trying to prove that any image (to a certain extent, mainly images captured on mobile devices) can be handled by my application.

I understand that some images may be so large that by loading them into memory will cause exceptions. This is something I will try and handle later.

In some cases the images that will be uploaded can span from 1 to 200.

I'm trying to look for the most optimal solution that will scale well.

Scrophulariaceous

If you want to use Mutipart Request with Volley, IMO, you can refer my sample code at github.com/ngocchung/VolleyNoApache/blob/master/app/src/main/…. In @kevinkl3 answer, https://mcmap.net/q/89355/-how-to-send-a-multipart-form-data-post-in-android-with-volley I also had an answer there – Bristle

Answer 1

I looked into using Volley as a mechanism to transport large JSON objects to a server and found this answer. This answer essentially proved that Volley would be a bad idea for what I wanted.

I switched to OkHttp and now use their streaming methods allowing the JSON to be streamed to the server and then read the response with a streamlined approach. I used the GSON library to parse the response as OKHttp allows the response JSON/Object to be streamed into a reader object which Gson then uses for internal streaming and parsing to an POJO class.

The ony reason why I did not switch to a Multi-Part request was due to the server side implementation being rigid and unchangeable to cover for Multi-Part requests, it strictly expected a JSON representation of data and files.

For handling Base64 Images on Android I severely reccomend not using the String representation and merely converting to Bytes to save on using an excessive amount of memory. I read this article on String memory usage and management. With the Bytes you may easily transport the data without leaving a massive footprint on memory.

For Displaying the images I still avoid the bytes to String conversion by using the Image library Glide. They allow you to pass in a byte[] which was of massive convenience.

Scrophulariaceous

Answer 2

... for multiple resolutions images (5620 x 3747 - 4.92MB or 3264 x 1836 - 1.35MB)...

Not sure if this is the file size or the memory needed to allocate the image in memory, but taking a look at the following link: http://www.scantips.com/basics1d.html, I see this:

For a 4000 x 2500 pixel image,

then: 4000 x 2500 pixels = 4000x2500 = 10 megapixels

4000x2500 x 3 = 30 million bytes (if 24-bit RGB)

30,000,000 bytes / (1024 x 1024) = 28.61 megabytes (MB)

This is simply how large the data is - For ANY 24-bit 10 megapixel image, but JPG files compress it smaller (only while in the file).

I think that the images you're handling are taking much more memory than you expect.

Also, taking a look at this question and answer: https://mcmap.net/q/167442/-what-is-the-effect-of-encoding-an-image-in-base64, we know that a base64 representation of an image take up to 37% more memory than the original image size.

How can I write the Base64 String in a stream to a JSON object that will contain the specific values needed to post into my server?

I think you could do this (with small images not large ones) by simply adding the base64 representation of the image in a JSON object then posting it to the server.

Would it be easier to change my server to accept a Multi-Part request instead of a JSON based POJO with multiple Base64 Strings?

In my opinion that would be your best option to implement what you're trying to achieve.

I currently use Volley and there isn't an official Multi-Part Request as well as IO streaming.

You can take a look at this answer: https://mcmap.net/q/89355/-how-to-send-a-multipart-form-data-post-in-android-with-volley, you can definitely can do it with volley, but if you want an alternative you can try with retrofit (http://square.github.io/retrofit/) they support Multipart our of the box.

eg:

@Multipart
@PUT("user/photo")
Call<User> updateUser(@Part("photo") RequestBody photo, @Part("description") RequestBody description); 

Standstill