Is it possible to render an Android View to an OpenGL FBO or texture?

Question

Is it possible to render a View (say, a WebView) to an FBO so it can be used as a texture in an OpenGL composition?

Croquette

Continuously? Or as a one-time snapshot sort of deal? – Kilmer

As far as I know, no. AFAIK best you can do is render View to Bitmap and assign it to texture using GLUtils. – Immorality

@Immorality I can't imagine the frame rate of doing that being very high. There must be a better way than that. – Croquette

Answer 1

Yes is it certainly possible, I have written up a how-to here; http://www.felixjones.co.uk/neo%20website/Android_View/

However for static elements that won't change, the bitmap option may be better.

Threadbare

I found not all views can be draw. TextureView can not be draw like this. Could you resolve it? – Subequatorial

@fadden If I use TextureView, I found onSurfaceTextureAvailable of TextureView.OnFrameAvailableListener not called back. Why can not TextureView drawed like this? – Subequatorial

Answer 2

I brought together a complete demo project which renders a view to GL textures in real time in an efficient way which can be found in this repo. It shows how to render WebView to GL texture in real time as an example.

Also a brief code for this can look like the following (taken from the demo project from the repo above):

public class GLWebView extends WebView {

    private ViewToGLRenderer mViewToGLRenderer;
    ...
    // drawing magic
    @Override
    public void draw( Canvas canvas ) {
        //returns canvas attached to gl texture to draw on
        Canvas glAttachedCanvas = mViewToGLRenderer.onDrawViewBegin();
        if(glAttachedCanvas != null) {
            //translate canvas to reflect view scrolling
            float xScale = glAttachedCanvas.getWidth() / (float)canvas.getWidth();
            glAttachedCanvas.scale(xScale, xScale);
            glAttachedCanvas.translate(-getScrollX(), -getScrollY());
            //draw the view to provided canvas
            super.draw(glAttachedCanvas);
        }
        // notify the canvas is updated
        mViewToGLRenderer.onDrawViewEnd();
    }

    ...
}


public class ViewToGLRenderer implements GLSurfaceView.Renderer{

    private SurfaceTexture mSurfaceTexture;
    private Surface mSurface;

    private int mGlSurfaceTexture;
    private Canvas mSurfaceCanvas;

    ...

    @Override
    public void onDrawFrame(GL10 gl){
        synchronized (this){
            // update texture
            mSurfaceTexture.updateTexImage();
        }
   }

    @Override
    public void onSurfaceChanged(GL10 gl, int width, int height){
        releaseSurface();
        mGlSurfaceTexture = createTexture();
        if (mGlSurfaceTexture > 0){
            //attach the texture to a surface.
            //It's a clue class for rendering an android view to gl level
            mSurfaceTexture = new SurfaceTexture(mGlSurfaceTexture);
            mSurfaceTexture.setDefaultBufferSize(mTextureWidth, mTextureHeight);
            mSurface = new Surface(mSurfaceTexture);
        }

    }

    public Canvas onDrawViewBegin(){
        mSurfaceCanvas = null;
        if (mSurface != null) {
            try {
                mSurfaceCanvas = mSurface.lockCanvas(null);
            }catch (Exception e){
                Log.e(TAG, "error while rendering view to gl: " + e);
            }
        }
        return mSurfaceCanvas;
    }

    public void onDrawViewEnd(){
        if(mSurfaceCanvas != null) {
            mSurface.unlockCanvasAndPost(mSurfaceCanvas);
        }
        mSurfaceCanvas = null;
    }
}

The demo output screenshot:

Degraded

Yeah it works. But the views still need to be added to Activity. Can we render them without adding them to the visible Layout? – Odum

This approach is problematic because the WebView now accepts all touch events from the screen. There's no layer for translation of the position of the actual touch event to a virtual touch event to the GL representation of the webview. – Beadle

@virus Is there a way to render the entire app, instead of one WebView? – Piling

@TimotejLeginus, I don't think so, but it depends on what you mean by the "entire app". You can render any view to GL and do with it what's suitable in your case. Also depending on your use case it might be helpful to use engines like Unity, AndEngine etc instead. – Degraded

Actually, I'm trying to pass the texture to Unity. I want to know if there's an easy method to render an entire app to a texture. – Piling

Answer 3

At least someone managed to render text this way:

Rendering Text in OpenGL on Android

It describes the method I used for rendering high-quality dynamic text efficiently using OpenGL ES 1.0, with TrueType/OpenType font files.

[...]

The whole process is actually quite easy. We generate the bitmap (as a texture), calculate and store the size of each character, as well as it's location on the texture (UV coordinates). There are some other finer details, but we'll get to that.

OpenGL ES 2.0 Version: https://github.com/d3kod/Texample2

Flossi