how to get bitmap information and then decode bitmap from internet-inputStream?

Question

background

suppose i have an inputStream that was originated from the internet of a certain image file.

i wish to get information about the image file and only then to decode it.

it's useful for multiple purposes, such as downsampling and also previewing of information before the image is shown.

the problem

i've tried to mark&reset the inputStream by wrapping the inputStream with a BufferedInputStream , but it didn't work:

inputStream=new BufferedInputStream(inputStream);
inputStream.mark(Integer.MAX_VALUE);
final BitmapFactory.Options options=new BitmapFactory.Options();
options.inJustDecodeBounds=true;
BitmapFactory.decodeStream(inputStream,null,options);
//this works fine. i get the options filled just right.

inputStream.reset();
final Bitmap bitmap=BitmapFactory.decodeStream(inputStream,null,options);
//this returns null

for getting the inputStream out of a url, i use:

public static InputStream getInputStreamFromInternet(final String urlString)
  {
  try
    {
    final URL url=new URL(urlString);
    final HttpURLConnection urlConnection=(HttpURLConnection)url.openConnection();
    final InputStream in=urlConnection.getInputStream();
    return in;
    }
  catch(final Exception e)
    {
    e.printStackTrace();
    }
  return null;
  }

the question

how can i make the code handle the marking an resetting ?

it works perfectly with resources (in fact i didn't even have to create a new BufferedInputStream for this to work) but not with inputStream from the internet...

---

EDIT:

it seems my code is just fine, sort of...

on some websites (like this one and this one), it fails to decode the image file even after reseting.

if you decode the bitmap (and use inSampleSize) , it can decode it fine (just takes a long time).

now the question is why it happens, and how can i fix it.

Moravian

Hey @android developer I am facing same error ... how did you resolve? please share your code... thanks in advance – Blanc

@DhirenParmar No. I didn't fix it. – Moravian

I've made a new suggestion for it here: code.google.com/p/android/issues/detail?id=231550 – Moravian

Answer 1

I believe the problem is that the call to mark() with the large value is overwritten by a call to mark(1024). As described in the documentation:

Prior to KITKAT, if is.markSupported() returns true, is.mark(1024) would be called. As of KITKAT, this is no longer the case.

This may be resulting in a reset() fail if reads larger than this value are being done.

Ingressive

didn't test it on kitkat, but i don't get what the docs say now. if is.mark(1024) isn't called, then what is? in any case, the problem i've reported still can't be handled like what i've written. the question now is: how do you do it correctly? – Moravian

This is the documentation of decodeStream. It means it calls is.mark(1024) internally, overwriting your call. How to do it correctly: reimplement decodeStream without the call to is.mark(1024). Else, you need to generate the input stream twice. – Ingressive

How do I do that? Simple copying from the source code won't be enough, since a lot of it is internal and private and I assume I also need the native code (C/C++) in this case... – Moravian

I'm not sure. I noticed that there's excessive encapsulation in the code, which makes it hard or impossible to reuse it (and that's a common theme both in android library and in java projects in general). If I went through this path, I'd fully copy and paste the whole BitmapFactory.java file, removing the problematic parts. I didn't try it this way, though, I went for generating the input stream twice. – Ingressive

i think it's better to download the file to the cache and then read from it instead of connecting to the server twice. about the encapsulation, you are correct. i have no idea why they made it so "secure". it's just bitmaps... – Moravian

Answer 2

(Here is a solution for the same problem, but when reading from disk. I didn't realize at first your question was specifically from a network stream.)

The problem with mark & reset in general here is that BitmapFactory.decodeStream() sometimes resets your marks. Thus resetting in order to do the actual read is broken.

But there is a second problem with BufferedInputStream: it can cause the entire image to be buffered in memory along side of where ever you are actually reading it into. Depending on your use case, this can really kill your performance. (Lots of allocation means lots of GC)

There is a really great solution here: https://mcmap.net/q/27100/-java-file-input-with-rewind-reset-capability

I modified it slightly for this particular use case to solve the mark & reset problem:

public class MarkableFileInputStream extends FilterInputStream
{
    private static final String TAG = MarkableFileInputStream.class.getSimpleName();

    private FileChannel m_fileChannel;
    private long m_mark = -1;

    public MarkableFileInputStream( FileInputStream fis )
    {
        super( fis );
        m_fileChannel = fis.getChannel();
    }

    @Override
    public boolean markSupported()
    {
        return true;
    }

    @Override
    public synchronized void mark( int readlimit )
    {
        try
        {
            m_mark = m_fileChannel.position();
        }
        catch( IOException ex )
        {
            Log.d( TAG, "Mark failed" );
            m_mark = -1;
        }
    }

    @Override
    public synchronized void reset() throws IOException
    {
        // Reset to beginning if mark has not been called or was reset
        // This is a little bit of custom functionality to solve problems
        // specific to Android's Bitmap decoding, and is slightly non-standard behavior
        if( m_mark == -1 )
        {
            m_fileChannel.position( 0 );
        }
        else
        {
            m_fileChannel.position( m_mark );
            m_mark = -1;
        }
    }
}

This won't allocate any extra memory during reads, and can be reset even if the marks have been cleared.

Mores

I'm not familiar with "FilterInputStream", but as it takes "FileInputStream" instead of a simple InputStream, this means that I need to give it a file that exists on the device, no? If not, did you check that it works? – Moravian

Correct, in my use case I take the network stream and save it to a disk cache. Then my actual image loader always uses a FileInputStream to read from the cache. FilterInputStream is simply a wrapper that allows you to transform an InputStream as it is read. FilterInputStream can take any InputStream, however MarkableFileInputStream is predicated on it being a FileInputStream due to the use of a FileChannel to do the mark & reset functionality. – Mores

I'd recommend using DiskLruCache if you're interested in caching the images. github.com/JakeWharton/DiskLruCache It can really help reduce redundant network traffic and load times one you've got your data the first time. – Mores

The thing is that the question was how to avoid using the disk and do it all using the inputStream, so this isn't much the answer (though it might be correct). In the end though, I decided to use the disk as it's easier and might be better anyway. What is the difference between using what you did and simply using BufferedInputStream? – Moravian

I put it in the answer, but BufferedInputStream can cause, at worst, an entire second copy of the data to be resident in memory while you're reading it. That's how BufferedInputStream provides the functionality it provides, by throwing the data into an intermediate Buffer. – Mores

I see, so it's better than using BufferedInputStream. But if you already have the file itself, you could just call BitmapFactory.decode twice : once with inJustDecodeBounds=true (to get the information of the image), and the second set it to false, to do the real downsampling. – Moravian

Answer 3

whether you can mark / reset a stream depends on the implementation of the stream. those are optional operations and aren't typically supported. your options are to read the stream into a buffer and then read from that stream 2x, or just make the network connection 2x.

the easiest thing is probably to write into a ByteArrayOutputStream,

ByteArrayOutputStream baos = new ByteArrayOutputStream();
int count;
byte[] b = new byte[...];
while ((count = input.read(b) != -1) [
  baos.write(b, 0, count);
}

now either use the result of baos.toByteArray() directly, or create a ByteArrayInputStream and use that repeatedly, calling reset() after consuming it each time.

ByteArrayInputStream bais = new ByteArrayInputStream(baos.toByteArray());

that might sound silly, but there's no magic. you either buffer the data in memory, or you read it 2x from the source. if the stream did support mark / reset, it'd have to do the same thing in it's implementation.

Camden

can't BufferedInputStream just hold the bytes that i will return to soon ? i don't wish to read the entire file or create a new connection since it misses the whole point - i already have the inputStream and its data is also available... – Moravian

it depend on the implementation of the stream that was used to construct the BufferedInputStream. does it support reset()? – Camden

it should, no? anyway, suppose i use your method, doesn't it mean i download the entire bytes of the file? – Moravian

no, it should not. what you have to understand is that a stream is, well, a stream, as opposed to a buffer. it does not inherently buffer or cache the data after it is read. that's the whole point of a stream. and yes, it means you have to download and store in memory the entire bytes of the file, or you have to read it 2x from the source. those are your options. i'm curious where you think the data would come from. it's either in memory, or read from the source again. there's no magic here. – Camden

is there an inputStream class that caches the bytes, so that i could read some bytes, and then return back to some position ? i could even tell it how much bytes to support to go back if needed... – Moravian

yes, there's a class. it's called a byte[], or if you need the data available in a stream form, a ByteArrayInputStream. you can see my example on how to use it for your purpose. – Camden

but your example downloads the entire data. i don't want that. i want to download bytes only when needed . for example: get 1000 bytes (no more than it) , return to the first byte, read all of the bytes. – Moravian

you are in control of how much data you download right? if you only want to read 1000 bytes, abort your read after 1000 bytes. – Camden

but then how could i go back and read them again (instead of re-downloading them) and the rest of the bytes? – Moravian

my friend, i gave complete example code of that already. see where i create the ByteArrayInputStream? that stream does support reset(). you can read / reset this as many times as you need. the data is buffered in memory and is not re-read from the source. you create the ByteArrayInputStream 1x then read it as many times as you want. – Camden

no, as i've already written, you code reads the entire file instead of reading the first bytes that i ask for. only after it has read the entire file, it can handle further requests. – Moravian

no problem. maybe i should update the question to become clearer? what do you think i should write? – Moravian

if you want to read 1k bytes and do something with them, put a counter in your read loop and stop and do whatever you want after 1k bytes. then if you decide you want to keep reading the stream, keep reading. or don't. that being said, there is no method on BitmapFactory that lets you pass it the first n bytes and decode meta data about the image. you have to give it the entire complete stream so your only option is what i posted. that being said, if you want to read / decode the first N bytes of the compressed image format yourself without BitmapFactory, you are welcome to do that. – Camden

no, that's not the idea i've talked about. i want to let whoever uses the inputStream to read from it , and later i wish to reset it back to the beginning. the 1k was just an example. also, about bitmapFactory, i don't think it reads the entire file just to get its information, because the information is stored in the header of the file. only the worst case scenario would be to get all of the bytes as you've written. – Moravian

your idea will not work. i can't say this any other way. you either store the bits in memory and make them available for reading and re-reading, or you re-open the stream each time. you cannot reset a network stream unless you wrap it with another stream impl that caches the bytes. you can think what you want about bitmap factory, but the point is, you don't know. if you think you can write a more efficient bitmap factory, by all means, go grab the source and have at it. that is way beyond the scope of what can be covered in a SO question. – Camden

now you get it. it's possible, and that's what you've written: "you wrap it with another stream impl that caches the bytes" . question is how i do it, and also how to do it efficiently. there are many streams that wrap other streams (like BufferedInputStream for example) , and it's ok to do such a thing. – Moravian

btw, the BufferedInputStream is supposed to give you the functionality of mark&reset, as written in the API : developer.android.com/reference/java/io/… . it supports marking, always. the java API even says it in the description: docs.oracle.com/javase/6/docs/api/java/io/… – Moravian

dude, i said it's possible in my first post and wrote the code for you there. you read the stream into a ByteArrayOutputStream, get the byte[] out of that and construct a ByteArrayInputStream, then read / reset that as many times as you like. if you don't need access to the underlying bytes as an array, you can use BufferedInputStream. – Camden

but how? my code didn't work even though the BufferedInputStream does support the mark&reset... that's why i've written the question. i already know i can read the entire data, but i don't want that since i wish to avoid re-reading data and having to wait for the whole data to be read. – Moravian

now it seems that the code works fine in case it's png, but not when it's jpg. weird. it means my code should work. now the question is why it doesn't work with jpg files. in fact, it seems it works with some jpgs, but can't work with others. here's the test url i've used that doesn't work: farm5.staticflickr.com/4031/4637562776_f955d89e3c_o.jpg – Moravian

seems to me that my code doesn't work only on some websites , like the staticflicker.com website, and even then, on some files inside it. i wonder what is the reason for this, and how i can fix it. – Moravian

Answer 4

Here is a simple method that always works for me :)

 private Bitmap downloadBitmap(String url) {
    // initilize the default HTTP client object
    final DefaultHttpClient client = new DefaultHttpClient();

    //forming a HttoGet request
    final HttpGet getRequest = new HttpGet(url);
    try {

        HttpResponse response = client.execute(getRequest);

        //check 200 OK for success
        final int statusCode = response.getStatusLine().getStatusCode();

        if (statusCode != HttpStatus.SC_OK) {
            Log.w("ImageDownloader", "Error " + statusCode +
                    " while retrieving bitmap from " + url);
            return null;

        }

        final HttpEntity entity = response.getEntity();
        if (entity != null) {
            InputStream inputStream = null;
            try {
                // getting contents from the stream
                inputStream = entity.getContent();

                // decoding stream data back into image Bitmap that android understands
                image = BitmapFactory.decodeStream(inputStream);


            } finally {
                if (inputStream != null) {
                    inputStream.close();
                }
                entity.consumeContent();
            }
        }
    } catch (Exception e) {
        // You Could provide a more explicit error message for IOException
        getRequest.abort();
        Log.e("ImageDownloader", "Something went wrong while" +
                " retrieving bitmap from " + url + e.toString());
    }

    return image;
}

Macaroon

Where is the part of getting the bitmap information (width,height,...) ? Where is the downsampling part? – Moravian