OutOfMemoryError although vm has enough free memory

Question

I am getting this weird OutOfMemoryError that occurs although the dalvikvm reports enough heap space. Logs:

12-09 14:16:05.527: D/dalvikvm(10040): GC_FOR_ALLOC freed 551K, 21% free 38000K/47687K, paused 173ms, total 173ms
12-09 14:16:05.527: I/dalvikvm-heap(10040): Grow heap (frag case) to 38.369MB for 858416-byte allocation
12-09 14:16:05.699: D/dalvikvm(10040): GC_FOR_ALLOC freed 6K, 21% free 38832K/48583K, paused 169ms, total 169ms
12-09 14:16:05.894: D/dalvikvm(10040): GC_FOR_ALLOC freed 103K, 20% free 38929K/48583K, paused 169ms, total 169ms
12-09 14:16:05.894: I/dalvikvm-heap(10040): Forcing collection of SoftReferences for 858416-byte allocation
12-09 14:16:06.074: D/dalvikvm(10040): GC_BEFORE_OOM freed 6K, 20% free 38922K/48583K, paused 182ms, total 182ms
12-09 14:16:06.074: E/dalvikvm-heap(10040): Out of memory on a 858416-byte allocation.
12-09 14:16:06.074: I/dalvikvm(10040): "AsyncTask #2" prio=5 tid=17 RUNNABLE
12-09 14:16:06.074: I/dalvikvm(10040):   | group="main" sCount=0 dsCount=0 obj=0x42013580 self=0x5f2a48d8
12-09 14:16:06.074: I/dalvikvm(10040):   | sysTid=10101 nice=10 sched=0/0 cgrp=apps/bg_non_interactive handle=1591062136
12-09 14:16:06.074: I/dalvikvm(10040):   | schedstat=( 7305663992 4216491759 5326 ) utm=697 stm=32 core=1
12-09 14:16:06.074: I/dalvikvm(10040):   at android.graphics.BitmapFactory.nativeDecodeStream(Native Method)
12-09 14:16:06.074: I/dalvikvm(10040):   at android.graphics.BitmapFactory.decodeStream(BitmapFactory.java:619)
12-09 14:16:06.074: I/dalvikvm(10040):   at android.graphics.BitmapFactory.decodeStream(BitmapFactory.java:691)

As you can see right before the outofmemory occurs the dalvikvm reports about 10mb free memory after gc. The allocation is for a 800k bitmap. I doubt that there is a race condition between gc and bitmap decoding here, because the reported free memory of dalvik didn't drop below 8mb free memory in all log statements of the last 20-30 seconds before the crash.

The problem occurs on a Samsung Galaxy Tab 2 10.1 running Android 4.1.2. I'm using a modified version of the ImageFetcher classes from the Google I/O app (2012), so I'm already doing stuff like inJustDecodeBounds when loading images to optimize sampleSize option.

As per documentation in Managing Bitmap Memory Android allocates Bitmap pixel data in the dalvik heap (since Android 3.0), so why does decoding the bitmap cause an outofmemory with 10mb free memory?

Has anyone seen this before or may have an idea what's happening?

EDIT: Per request here is the image loading code from the Google I/O app 2012. In my app I am just calling

mImageFetcher.loadImage(myUrl, myImageView);

EDIT2: The relevant image decoding methods extracted from above link to make clear that I am already using sample size optimizations:

public static Bitmap decodeSampledBitmapFromDescriptor(
        FileDescriptor fileDescriptor, int reqWidth, int reqHeight) {

    // First decode with inJustDecodeBounds=true to check dimensions
    final BitmapFactory.Options options = new BitmapFactory.Options();
    options.inJustDecodeBounds = true;
    BitmapFactory.decodeFileDescriptor(fileDescriptor, null, options);

    // Calculate inSampleSize
    options.inSampleSize = calculateInSampleSize(options, reqWidth,
            reqHeight);

    // Decode bitmap with inSampleSize set
    options.inJustDecodeBounds = false;
    return BitmapFactory
            .decodeFileDescriptor(fileDescriptor, null, options);
}

public static int calculateInSampleSize(BitmapFactory.Options options,
        int reqWidth, int reqHeight) {
    // Raw height and width of image
    final int height = options.outHeight;
    final int width = options.outWidth;
    int inSampleSize = 1;

    if (height > reqHeight || width > reqWidth) {
        // Calculate ratios of height and width to requested height and
        // width
        final int heightRatio = Math.round((float) height
                / (float) reqHeight);
        final int widthRatio = Math.round((float) width / (float) reqWidth);

        // Choose the smallest ratio as inSampleSize value, this will
        // guarantee
        // a final image with both dimensions larger than or equal to the
        // requested height and width.
        inSampleSize = heightRatio < widthRatio ? heightRatio : widthRatio;

        // This offers some additional logic in case the image has a strange
        // aspect ratio. For example, a panorama may have a much larger
        // width than height. In these cases the total pixels might still
        // end up being too large to fit comfortably in memory, so we should
        // be more aggressive with sample down the image (=larger
        // inSampleSize).

        final float totalPixels = width * height;

        // Anything more than 2x the requested pixels we'll sample down
        // further.
        final float totalReqPixelsCap = reqWidth * reqHeight * 2;

        while (totalPixels / (inSampleSize * inSampleSize) > totalReqPixelsCap) {
            inSampleSize++;
        }
    }
    return inSampleSize;
}

Scrutinize

Could you share the code so we can help you ? – Folketing

you need to recycle ur bitmap – Veto

@Folketing added the source link – Scrutinize

@ShakeebAyaz recycle only frees native memory and the documentation says this only applies for Android versions below 3.0. Please correct me if the documentation is wrong. – Scrutinize

https://mcmap.net/q/1045355/-bitmap-recycle-with-largeheap-enabled – Eleemosynary

@LalitPoptani thanks, I'm already optimizing sample size. I have updated the question to make this clearer. The core question I have is "Why do I get an outofmemory when dalvik reports 10MB free memory?" – Scrutinize

You should try to provide some more info about your whole app. It's not really possible that the system is killing your app with just 38 Mb occupied on a tablet. Are you sure you are not using some native memory? The limit of your application is unique for Native memory + java heap. Are you using some native libraries? Maybe opencv or gmaps? Try to trace both java total memory with MAT and native memory usage wwboy6.wordpress.com/2012/12/09/android-ndk-memory-analyze – Bratton

@PasqualeAnatriello I am using Google Maps in the app. Does gmaps have it's own native memory? I will have a look at analyzing the native memory, thanks for the link. That would certainly answer the question, but what would the impact be on my app? I guess I can't do very much about gmaps allocating a lot of memory, so I'm probably back to optimizing my memory usage? – Scrutinize

Unfortunately gmaps uses a lot of memory. Depending on screen size it could occupy up to 20 mb of heap space. Most of the memory gmaps uses is indeed in the native part. The only ways to go are: 1) optimize as much as you can your Bitmap usage 2) go with the large heap flag – Bratton

Answer 1

About OutOfMemory (OOM) on ~850000 bytes allocation when you have 10 Mb free, that is most certainly due to memory fragmentation there is no guarantee that the heap has one continous chunk of memory bigger than 850000 bytes and this is why you get the OOM.

Seems strange that you still get the error, you seem to have done some optimizations already, do you really release all memory you hold? I mean you have 38 Mb used heap, what are contained in that memory?

Have you tried looking at image loading libraries, such as, for instance picasso ?

Where one could such things as: Picasso.with(context).load("https://i.sstatic.net/E5w9Z.jpg").fit().into(imageView);

(This downloads and caches the image and fits and draws into an imageView, neat!)

Update

1. You should analyze your hprof file (should be available on root sdcard path since you got an OOM) in MAT to see if your holding on to unnecessary references
2. Release those references (null them out to let the GC collect them)
3. Use inBitmap to reuse memory (became more powerful in KitKat, images does not need to be of same size as the previous image, just hold as much or less memory than previous)
4. If you frequently ask for same image consider caching with e.g. LruCache
5. If they are really big bitmaps try tile the image (will load small square bitmaps into a big image) look at: TileImageView (this is using GlView to draw though..)

Banda

Thanks. Memory fragmentation sounds like a reasonable explanation for such a behavior. I have looked at Picasso and use it in another project, but what we are using is basically the image loading part of Volley and provides similar functionality. We will probably add inBitmap loading in one of the next product iterations, which should hopefully be another memory optimization, but should especially help if it's really a fragmentation issue. If you have any (semi-) official links about memory fragmentation on Android I will gladly award the bounty to you. Thanks again. – Scrutinize

memory fragmentation talks about what happens when we allocate big objects, the problem is that the small objects laying around in the heap pollutes the heap, preventing allocation of big objects eventually. It has been discussed in many other threads here on SO as well. One strategy you could use to prevent the OOM from happening, is to pool your bitmaps (keep the big memory instead of reallocating the same chunk again and again). – Banda

In my case it is probably a combination of Google Maps allocating a load of native heap that does not show up in the heap dump and perhaps also a fragmentation issue. Since Pasquale only posted a comment and not an answer I'l award the bounty to you. Thanks for your input. – Scrutinize

Answer 2

It seems like ICS and later Android versions are not letting your VM go to the total size of the heap. I've seen the same thing in my app,

You can add

android:largeHeap="true" 

to your <application> which gives your app a much larger heap. Not nice but works...

Nelrsa

Thanks for your answer. Are there any drawbacks to setting largeHeap to true? Will the app still install and run smoothly on devices with low memory? Do you know of any backwards compatibility issues down to 2.2? I have researched a bit about it, but haven't found anything conclusive. – Scrutinize

I tested it on emulators down to API level 8 and on Apkudo and it works. I didn't roll out this version, though. If the attribute is unknown at a certain API level it will simply be ignored. If you are using a shared user ID, you need to roll out all apps at once since the one that starts the process rules. I noticed that on KitKat my app ran into OutOfMemoryExceptions for no reason and the flag made this go away with no other noticeable effect. I read somewhere that KitKat would shut down apps which consume too much memory so the largeHeap flag probably just disables this "feature". – Nelrsa

Apart from this, my gut instinct is telling me there is some residual risk of getting punished for this one day :-S – Nelrsa

Thanks. I'm still reluctant to use this. I only see the problem on a couple of devices and fear I might be creating other problems (e.g. performance issues) on other devices. I'll keep the question open for a couple more days to see if someone else joins the discussion, but will accept yours if no other valid explanation is offered. Thanks. – Scrutinize

Using large heap is strongly discouraged unless you really need to (ex you have to manage big images on tablets or similar stuff). If you are having OOM errors you should really try to optimize the way you are using images like scaling them down to the size you need them to be or reducing the images cache (if you got one) – Bratton

That's really not solving the problem, only hiding it (a bit longer) – Banda

I was hoping to get some more insight about this option from our discussion. I could not find any documentation that "strongly discourages" this option (or I would wonder why it is there at all...). Your heap seems to be fragmented (maybe a heap dump and MAT can prove it) and it looks like the VM is not willing to allocate a larger heap. If you really need this amount of memory, use largeHeap or start a service or activity in a [separate process]( developer.android.com/guide/topics/manifest/…) – Nelrsa

It's a valid option when you have runned out of ideas, but the first steps should be to analyze memory usage, try to reuse the memory already allocated (through pooling the objects), and assist GC by nulling unused objects, alternatively recycle bitmaps as well. When all that fails then you might try largeHeap, but it's still no guarantee that the issue will not surface again (hence hiding the problem a bit longer). Also with largeHeap there's risk for the ActivityManager to kill of some services/activities which it thinks is unnecessary, but might still actually be quite valid to keep. – Banda

Could you elaborate on your last statement? How is largeHeap affecting the behaviour of the ActivityManager? – Nelrsa

Some quotes from Google I/O 2011: Memory management for Android Apps : "This is not what you should do just because you get an OOM" "bigger heap more time in GC" "other apps get kicked out of memory" "poorer performance". From about 6:55 in that talk to 7:30 – Banda

No, I mean your statement about the ActivityManager changing its behaviour based on this flag. – Nelrsa

Since i quoted "other apps get kicked out of memory" above, and ActivityManager is responsible for just that: activities / services then it explains itself? – Banda

Answer 3

try with this

public static Bitmap decodeFile(File f,int WIDTH,int HIGHT){
        try {
            //Decode image size
            BitmapFactory.Options o = new BitmapFactory.Options();
            o.inJustDecodeBounds = true;
            BitmapFactory.decodeStream(new FileInputStream(f),null,o);

            //The new size we want to scale to
            final int REQUIRED_WIDTH=WIDTH;
            final int REQUIRED_HIGHT=HIGHT;
            //Find the correct scale value. It should be the power of 2.
            int scale=1;
            while(o.outWidth/scale/2>=REQUIRED_WIDTH && o.outHeight/scale/2>=REQUIRED_HIGHT)
                scale*=2;

            //Decode with inSampleSize
            BitmapFactory.Options o2 = new BitmapFactory.Options();
            o2.inSampleSize=scale;
            return BitmapFactory.decodeStream(new FileInputStream(f), null, o2);
        }
            catch (FileNotFoundException e) {}
        return null;
    }

this will scale bitmap as per width and height you pass. this function finds correct scale as per image resolution.

Acinaciform

Answer 4

This code will be help full you to, try this

public Bitmap decodeSampledBitmapFromResource(String path,
        int reqWidth, int reqHeight) {

    // First decode with inJustDecodeBounds=true to check dimensions
    final BitmapFactory.Options options = new BitmapFactory.Options();
    options.inJustDecodeBounds = true;
    BitmapFactory.decodeFile(path, options);
    // BitmapFactory.decodeResource(getResources(), id)
    // Calculate inSampleSize
    options.inSampleSize = calculateInSampleSize(options, reqWidth,
            reqHeight);

    // Decode bitmap with inSampleSize set
    options.inJustDecodeBounds = false;
    return BitmapFactory.decodeFile(path, options);
}

public int calculateInSampleSize(BitmapFactory.Options options,
        int reqWidth, int reqHeight) {
    // Raw height and width of image
    final int height = options.outHeight;
    final int width = options.outWidth;
    int inSampleSize = 1;

    if (height > reqHeight || width > reqWidth) {

        // Calculate ratios of height and width to requested height and
        // width
        final int heightRatio = Math.round((float) height
                / (float) reqHeight);
        final int widthRatio = Math.round((float) width / (float) reqWidth);

        // Choose the smallest ratio as inSampleSize value, this will
        // guarantee
        // a final image with both dimensions larger than or equal to the
        // requested height and width.
        inSampleSize = heightRatio < widthRatio ? heightRatio : widthRatio;
    }

    return inSampleSize;
}

Yeargain

try these #15061864 – Yeargain

wiki.eclipse.org/… – Yeargain

thanks again, but the error occurs on the device. Increasing the Eclipse heap size won't help I'm afraid. Increasing the heap available for the apps on the device is not possible afaik. – Scrutinize

try this #478072 – Yeargain

thanks, but please re-read the question. I'm already doing sample size optimizations, but I get an outofmemory error even though the VM reports 10MB free memory. – Scrutinize

Answer 5

I am a newbie I am not sure but try sampling the image like this

  final BitmapFactory.Options options = new BitmapFactory.Options();
  options.inSampleSize = 8;

  Bitmap bm=BitmapFactory.decodeFile(strPath,options);

Use inSampleSize to load scales bitmaps to memory. Using powers of 2 for inSampleSize values is faster and more efficient for the decoder. However, if you plan to cache the resized versions in memory or on disk, it’s usually still worth decoding to the most appropriate image dimensions to save space.

Veto

thanks. I'm already doing sample size optimizations. edited my question to make this more clear. – Scrutinize

Answer 6

create Imageloader class

package com.example.model;

import java.io.File;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.FileOutputStream;
import java.io.InputStream;
import java.io.OutputStream;
import java.net.HttpURLConnection;
import java.net.URL;
import java.util.Collections;
import java.util.Map;
import java.util.WeakHashMap;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;

import android.app.Activity;
import android.content.Context;
import android.graphics.Bitmap;
import android.graphics.BitmapFactory;
import android.graphics.Matrix;
import android.widget.ImageView;

public class ImageLoader {

    MemoryCache memoryCache=new MemoryCache();
    FileCache fileCache;
    private Map<ImageView, String> imageViews=Collections.synchronizedMap(new WeakHashMap<ImageView, String>());
    ExecutorService executorService; 

    public ImageLoader(Context context){
        fileCache=new FileCache(context);
        executorService=Executors.newFixedThreadPool(5);
    }

    final int stub_id=R.drawable.load;
    public void DisplayImage(String url, ImageView imageView)
    {
        imageViews.put(imageView, url);
        Bitmap bitmap=memoryCache.get(url);
        if(bitmap!=null)
            imageView.setImageBitmap(bitmap);
        else
        {
            queuePhoto(url, imageView);
            imageView.setImageResource(stub_id);
        }
    }

    private void queuePhoto(String url, ImageView imageView)
    {
        PhotoToLoad p=new PhotoToLoad(url, imageView);
        executorService.submit(new PhotosLoader(p));
    }

    private Bitmap getBitmap(String url) 
    {
        File f=fileCache.getFile(url);

        //from SD cache
        Bitmap b = decodeFile(f);
        if(b!=null)
            return b;

        //from web
        try {
            Bitmap bitmap=null;
            URL imageUrl = new URL(url);
           // System.out.println("url :"+imageUrl);
            HttpURLConnection conn = (HttpURLConnection)imageUrl.openConnection();
            conn.setConnectTimeout(30000);
            conn.setReadTimeout(30000);
            conn.setInstanceFollowRedirects(true);

            InputStream is=conn.getInputStream();
            OutputStream os = new FileOutputStream(f);
            Utils.CopyStream(is, os);

            os.close();
            bitmap = decodeFile(f);


            return bitmap;
        } catch (Throwable ex){
           ex.printStackTrace();
           if(ex instanceof OutOfMemoryError)
               memoryCache.clear();
           return null;
        }
    }

    //decodes image and scales it to reduce memory consumption
    private Bitmap decodeFile(File f){
        try {
            //decode image size
            BitmapFactory.Options o = new BitmapFactory.Options();
            o.inJustDecodeBounds = true;
            o.inPurgeable = true; // Tell to garbage collector that whether it needs free memory, the Bitmap can be cleared
            o.inTempStorage = new byte[32 * 1024];
            BitmapFactory.decodeStream(new FileInputStream(f),null,o);

            //Find the correct scale value. It should be the power of 2.
            final int REQUIRED_SIZE=70;
            int width_tmp=o.outWidth, height_tmp=o.outHeight;
            int scale=1;
            while(true){
                if(width_tmp/2<REQUIRED_SIZE || height_tmp/2<REQUIRED_SIZE)
                    break;
                width_tmp/=2;
                height_tmp/=2;
                long heapSize = Runtime.getRuntime().maxMemory();

               long heapsize1=(heapSize/(1024*1024));
               if(heapsize1>95)
               {
                  scale*=1;
                 // System.out.println("scale1 :");
               }else if(heapsize1>63 && heapsize1<=95){
                  scale*=2;
                // System.out.println("scale2 :");
               }else if(heapsize1>31 && heapsize1<=63){
                   scale*=2;
                 // System.out.println("scale22 :");
               }else if(heapsize1>0 && heapsize1<=31){
                      scale*=2;
                    // System.out.println("scale23 :");
                   }
               /*else if(heapsize1>31 && heapsize1<=63){
                  scale*=2;
                // System.out.println("scale2 :");
               }else if(heapsize1>0 && heapsize1<=31){
                  scale*=2;
                    // System.out.println("scale2 :");
                   }*/

            }

            //decode with inSampleSize
            BitmapFactory.Options o2 = new BitmapFactory.Options();
            o2.inPurgeable = true; // Tell to garbage collector that whether it needs free memory, the Bitmap can be cleared
            o2.inTempStorage = new byte[32 * 1024];
            o2.inSampleSize=scale;
            Bitmap bitmap1=BitmapFactory.decodeStream(new FileInputStream(f), null, o2);
          //  System.out.println("width : "+bitmap1.getWidth()+ " height : "+bitmap1.getHeight());
       /*     if(bitmap1.getHeight()>=bitmap1.getWidth())
            {

                bitmap1 = Bitmap.createScaledBitmap(bitmap1, bitmap1.getHeight(),bitmap1.getWidth(), true);
            }else{
                //bmp = Bitmap.createScaledBitmap(bmp, (int) height2,width, true);
                Matrix matrix = new Matrix();

                matrix.postRotate(270);
                bitmap1 = Bitmap.createBitmap(bitmap1 , 0, 0, bitmap1 .getWidth(), bitmap1 .getHeight(), matrix, true);

            }*/
            return bitmap1;
        } catch (FileNotFoundException e) {}
        return null;
    }

    //Task for the queue
    private class PhotoToLoad
    {
        public String url;
        public ImageView imageView;
        public PhotoToLoad(String u, ImageView i){
            url=u; 
            imageView=i;
        }
    }

    class PhotosLoader implements Runnable {
        PhotoToLoad photoToLoad;
        PhotosLoader(PhotoToLoad photoToLoad){
            this.photoToLoad=photoToLoad;
        }

        @Override
        public void run() {
            if(imageViewReused(photoToLoad))
                return;
            Bitmap bmp=getBitmap(photoToLoad.url);
            memoryCache.put(photoToLoad.url, bmp);
            if(imageViewReused(photoToLoad))
                return;
            BitmapDisplayer bd=new BitmapDisplayer(bmp, photoToLoad);
            Activity a=(Activity)photoToLoad.imageView.getContext();
            a.runOnUiThread(bd);
        }
    }

    boolean imageViewReused(PhotoToLoad photoToLoad){
        String tag=imageViews.get(photoToLoad.imageView);
        if(tag==null || !tag.equals(photoToLoad.url))
            return true;
        return false;
    }

    //Used to display bitmap in the UI thread
    class BitmapDisplayer implements Runnable
    {
        Bitmap bitmap;
        PhotoToLoad photoToLoad;
        public BitmapDisplayer(Bitmap b, PhotoToLoad p){
            bitmap=b;photoToLoad=p;
            }
        public void run()
        {
            if(imageViewReused(photoToLoad))
                return;
            if(bitmap!=null)
                photoToLoad.imageView.setImageBitmap(bitmap);
            else
                photoToLoad.imageView.setImageResource(stub_id);
        }
    }

    public void clearCache() {
        memoryCache.clear();
        fileCache.clear();
    }

}

use that

or use that link lazy loader example

Leg

Answer 7

First of all, all answers will help you to low down your memory consumption, but no one of us can really help with your App because we don't know the entire code.

I'll share with you an experience that I've had with our App. We were experiencing too many OOMs, and we've tried Picasso, BitmapFun example from Android, and finally I've decided to profile my App.

Android Studio provides you a Tool called Monitor, where you can track how much memory does your App allocate in RAM in every Activity you call, in every rotation, etc.

You can export the profiling that you've just done using HEAP Dump, and then import it on Eclipse Memory Analyzer Tool. There you can run an Object Leaks Detector (or something like that. It will help you showing which Objects are leaked in RAM.

I.E. We were using old method (OnRetainCustomConfig...) where you can just store the reference of your objects. The problem was that the new Activity received the references of the old Activity's Object and because of that, GC didn't clean succesfully old Activities because it assumed that they were still in use.

I hope my comment helps you

Letitialetizia

You are right of course. And I wasn't specifically looking for a solution to the problem, but rather I want to understand why this strange outofmemory gets thrown even though the VM reports 10MB free memory. In my case I think it's probably a combination of Google Maps allocating native memory that does not show up in the heap dumps and perhaps also memory fragmentation, so the discussion in this thread was already very helpful. Thanks for your answer. – Scrutinize

Give a try to Eclipse MAT, there you can get a Leaks Suspects Report, its better than trying to guess – Letitialetizia