SurfaceFlinger/gralloc Out of Memory error when allocating buffer memory for use with virtual frame buffer on Android x86 Nougat

Question

We're customizing a fork of the Android x86 Nougat (Android v7.1) project. Because there is no video display hardware in our system we're trying to setup the virtual frame buffer (VFB) so we can then handle the video display in a custom manner (e.g. route the video display through a VNC server etc).

We when the system boots up SurfaceError tries to initialize it fails when it attempts to allocate memory using gralloc. The following is an extract of logcat:

03-02 18:31:21.757   986   986 E gralloc : #### gralloc_alloc
03-02 18:31:21.757   986   986 E gralloc : #### gralloc_alloc_framebuffer
03-02 18:31:21.757   986   986 E gralloc : #### gralloc_alloc_framebuffer_locked
03-02 18:31:21.757   986   986 E Gralloc1On0Adapter: gralloc0 allocation failed: -12 (Out of memory)
03-02 18:31:21.757   986   986 E GraphicBufferAllocator: Failed to allocate (360 x 480) format 5 usage 6656: 5
03-02 18:31:21.757   986   986 V Gralloc1On0Adapter: Destroying descriptor 3
03-02 18:31:21.757   986   986 V Gralloc1On0Adapter: dump(0 (0xbfe341f4), 0x0
03-02 18:31:21.757   986   986 V Gralloc1On0Adapter: dump(0 (0xbfe341f4), 0x0
03-02 18:31:21.757   986   986 D GraphicBufferAllocator: Allocated buffers:
03-02 18:31:21.757   986   986 D GraphicBufferAllocator: 0xab5f60c0:  675.00 KiB |  360 ( 360) x  480 |        5 | 0x00001a00 | FramebufferSurface
03-02 18:31:21.757   986   986 D GraphicBufferAllocator: 0xab5f60f0:  675.00 KiB |  360 ( 360) x  480 |        5 | 0x00001a00 | FramebufferSurface
03-02 18:31:21.757   986   986 D GraphicBufferAllocator: Total allocated (estimate): 1350.00 KB
03-02 18:31:21.757   986   986 E         : GraphicBufferAlloc::createGraphicBuffer(w=360, h=480) failed (Out of memory), handle=0x0
03-02 18:31:21.757   986   986 E BufferQueueProducer: [FramebufferSurface] allocateBuffers: failed to allocate buffer (0 x 0, format 5, usage 0)

Here's links to the full logs:

Init log (dmesg): http://pastebin.com/sr1rAK43

Logcat: http://pastebin.com/SVYhvgu5

What we've done to get things running:

1. In kernel/arch/x86/configs/android-x86_defconfig set the option: CONFIG_FB_VIRTUAL to M for VFB module mode 



2. In kernel/drivers/video/fbdev/vfb.c updated frame buffer size up to 64MB: #define VIDEOMEMSIZE (64*1024*1024)



3. In device/generic/common/init.sh I start VFB and disable UVESA frame buffer
ing:

function do_init()

#
init_hal_gralloc

modprobe vfb vfb_enable=1



And in the same init.sh at the file end before "return 0":

/system/bin/fbset 360 480 20


I'd like to know:

1) How I can resolve the out of memory error?


2) How you enable vfb from kernel command line?


3) Any configurations/setups I may have missed in getting VFB to work on kernel 4.4 in Android 7.x

Mesomorph

Do you have access to the Manifest and/or build.gradle? If yes you can declare a largeHeap to get more memory and avoid the exception – Inwardly

It's not a normal Java Android app that I'm trying to modify, it's low-level services running at the OS level. – Mesomorph

The logs contain lines like "fbset : type=1400 audit(0.0:33): avc: denied { write }". Does fbset work like you expected ? – Chili

Any luck in getting answer to this? I am facin the same error while playing a 4k content on firestick using exoplayer – Dovecote

@Dovecote Sorry I can't remember, it was a few years ago. – Mesomorph