Flutter C++ Memory allocation causes jank on raster thread - Android NDK Dart FFI

Question

I have a flutter app which uses Dart ffi to connect to my custom C++ audio backend. There I allocate around 10MB of total memory for my audio buffers. Each buffer has 10MB / 84 of memory. I use 84 audio players. Here is the ffi flow:

C++ bridge:

extern "C" __attribute__((visibility("default"))) __attribute__((used))
void *
loadMedia(char *filePath, int8_t *mediaLoadPointer, int64_t *currentPositionPtr, int8_t *mediaID) {
    LOGD("loadMedia %s", filePath);

    if (soundEngine == nullptr) {
        soundEngine = new SoundEngine();
    }

    return soundEngine->loadMedia(filePath, mediaLoadPointer, currentPositionPtr, mediaID);
}

In my sound engine I launch a C++ thread:

void loadMedia(){

    std::thread{startDecoderWorker,
                    buffer,
    }.detach();
 }

void startDecoderWorker(float*buffer){
     buffer = new float[30000]; // 30000 might be wrong here, I entered a huge value to just showcase the problem, the calculation of 10MB / 84 code is redundant to the code
}

So here is the problem, I dont know why but when I allocate memory with new keyword even inside a C++ thread, flutters raster thread janks and I can see that my flutter UI janks lots of frames. This is also present in performance overlay as it goes all red for 3 to 5 frames with each of it taking around 30 40ms. Tested on profile mode.

Here is how I came to this conclusion: If I instantly return from my startDecoderWorker without running new memory allocation code, when I do this there is 0 jank. Everything is smooth 60fps, performance overlay doesnt show me red bars.

Here are some screenshots from Profile mode:

Fadeless

Have you tried just doing your memory allocation at program startup instead of thread startup? If the flutter thread is doing any sort of memory allocation, I could see there being some sort of lock contention in malloc: #4600708 Based on what I'm reading there, your threads could block for a long time while the allocator finds enough contiguous blocks to meet your request, which could block new/malloc in your flutter thread. – Aerometeorograph

I could do the thing you said but it would cause lots of logic since I keep on creating deleting audio players. This would require a lot of work for me and it seems like an anti pattern – Fadeless

For real-time software, it absolutely is NOT an anti-pattern. – Aerometeorograph

What if I suddenly need more memory? It would cause another jank this is why I said it's an anti pattern, I might be wrong. So it appears that my the best possibility for me to do this is to allocate memory at the very beginning of my app – Fadeless

Can you give a minimal reproducible sample (e.g. on GitHub)? Maybe it is not a problem for your new float[30000], but caused by something else in your code! – Celik

In addition, 10MB/84 sounds like a tiny thing. I myself deal with images in my app, and it is often a few megabytes (you know, uncompressed bitmaps). I do not see any junk. – Celik

Actually your "If I instantly return from my startDecoderWorker without running new memory allocation..." logic may be wrong. If you directly return, that is an empty function, and C++ can optimize heavily on it. There are thousands of possible reasons without seeing your reproducible minimal sample. – Celik

I tried to set the buffer size as 1 to test what you said. No jank. I think this is also another proof – Fadeless

i will provide an example soon I myself deal with images in my app, and it is often a few megabytes (you know, uncompressed bitmaps). did u use flutter for this? – Fadeless

@Celik added profile mode debug screenshots, I tried to create a simple project but I could not reproduce the jank. In the demo project In debug mode I was able to see performance hits but in profile mode i saw 0 hits. Bear in mind that my app is a complicated app, this bug is maybe due to several other things adding up BUT HOWEVER if I allocate only 2 floats instead of 30000 there is 0 jank. This should be enough to justify that there is a problem with mem alloc – Fadeless

@csguy that may not be a problem caused by 30000. for example, if you do something like for(int i=0;i<float_array_length;++i) then the slow can be caused by that. – Celik

did u use flutter for this? -> c++ code (opencv), and use ffi to communicate w/ flutter – Celik

@Celik you are a genius! I solved the issue, the problem was that its not because im allocating memory, its because as soon as I allocate memory my audio decoder starts decoding and fills the buffer. So apparently, its the decoder thats making everything lag. Even tho its in a seperate thread... its all clear now, thank you for chatting and helping me, make a comment and i will accept ur answer! – Fadeless

@csguy Sooooooooo happy to hear that! – Celik

Answer 1

The actual cause, after discussions (in the comments of the question), is not because the memory allocation is too slow, but lie somewhere else - the calculations which will be heavy if the allocation is big.

For details, please refer to the comments and discussions of the question ;)

Celik