How to pass native void pointers to a Dart Isolate - without copying?

Question

I am working on exposing an audio library (C library) for Dart. To trigger the audio engine, it requires a few initializations steps (non blocking for UI), then audio processing is triggered with a perform function, which is blocking (audio processing is a heavy task). That is why I came to read about Dart isolates.

My first thought was that I only needed to call the performance method in the isolate, but it doesn't seem possible, since the perform function takes the engine state as first argument - this engine state is an opaque pointer ( Pointer in dart:ffi ). When trying to pass engine state to a new isolate with compute function, Dart VM returns an error - it cannot pass C pointers to an isolate.

I could not find a way to pass this data to the isolate, I assume this is due to the separate memory of main isolate and the one I'm creating.

So, I should probably manage the entire engine state in the isolate which means :

• Create the engine state
• Initialize it with some options (strings)
• trigger the perform function
• control audio at runtime

I couldn't find any example on how to perform this actions in the isolate, but triggered from main thread/isolate. Neither on how to manage isolate memory (keep the engine state, and use it). Of course I could do

Here is a non-isolated example of what I want to do :

Pointer<Void> engineState = createEngineState();
initEngine(engineState, parametersString);
startEngine(engineState);
perform(engineState);

And at runtime, triggered by UI actions (like slider value changed, or button clicked) :

setEngineControl(engineState, valueToSet);
double controleValue = getEngineControl(engineState);

The engine state could be encapsulated in a class, I don't think it really matters here. Whether it is a class or an opaque datatype, I can't find how to manage and keep this state, and perform triggers from main thread (processed in isolate). Any idea ?

In advance, thanks.

PS: I notice, while writing, that my question/explaination may not be precise, I have to say I'm a bit lost here, since I never used Dart Isolates. Please tell me if some information is missing.

EDIT April 24th : It seems to be working with creating and managing object state inside the Isolate. But the main problem isn't solved. Because the perform method is actually blocking while it is not completed, there is no way to still receive messages in the isolate. An option I thought first was to use the performBlock method, which only performs a block of audio samples. Like this :

while(performBlock(engineState)) {
// listen messages, and do something
}

But this doesn't seem to work, process is still blocked until audio performance finishes. Even if this loop is called in an async method in the isolate, it blocks, and no message are read.

I now think about the possibility to pass the Pointer<Void> managed in main isolate to another, that would then be the worker (for perform method only), and then be able to trigger some control methods from main isolate.

The isolate Dart package provides a registry sub library to manage some shared memory. But it is still impossible to pass void pointer between isolates.

[ERROR:flutter/lib/ui/ui_dart_state.cc(157)] Unhandled Exception: Invalid argument(s): Native objects (from dart:ffi) such as Pointers and Structs cannot be passed between isolates.

Has anyone already met this kind of situation ?

Purusha

Can't you just do the whole process is the isolate? (Create the pointer, and communicate with the main isolate with a Recieve/Send Port) – Insubstantial

Yes this is finally what I did, and it seems to work fine. But for design reason, it seemed strange to me to trigger with message strings, with a condition in the isolate side. But well, it works. – Purusha

how do you call dart:ffi native function from isolate? I'm getting the following error when trying to do so: NoSuchMethodError (NoSuchMethodError: The method 'FfiTrampoline' was called on null. Receiver: null Tried calling: FfiTrampoline()) – Joslin

I call it the normal way. Just like another function. Are you sure you're calling the right function, not the pointer function instead ? – Purusha

@Purusha do you have any examples of the code you ended up with? as I'm trying to do something very similar with a Dart FFI binding to an audio library as well. – Volpe

@Volpe I posted the answer below – Purusha

Answer 1

It is possible to get an address which this Pointer points to as a number and construct a new Pointer from this address (see Pointer.address and Pointer.fromAddress()). Since numbers can freely be passed between isolates, this can be used to pass native pointers between them.

In your case that could be done, for example, like this (I used Flutter's compute to make the example a bit simpler but that would apparently work with explicitly using Send/ReceivePorts as well)

// Callback to be used in a backround isolate.
// Returns address of the new engine.
int initEngine(String parameters) {
  Pointer<Void> engineState = createEngineState();
  initEngine(engineState, parameters);
  startEngine(engineState);
  return engineState.address;
}

// Callback to be used in a backround isolate.
// Does whichever processing is needed using the given engine.
void processWithEngine(int engineStateAddress) {
  final engineState = Pointer<Void>.fromAddress(engineStateAddress);
  process(engineState);
}

void main() {
  // Initialize the engine in a background isolate.
  final address = compute(initEngine, "parameters");
  final engineState = Pointer<Void>.fromAddress(address);

  // Do some heavy computation in a background isolate using the engine.
  compute(processWithEngine, engineState.address);
}

Quoits

Answer 2

I ended up doing the processing of callbacks inside the audio loop itself.

while(performAudio())
{
      tasks.forEach((String key, List<int> value) {
        double val = getCallback(key);
        value.forEach((int element) {
          callbackPort.send([element, val]);
        });
      });

}

Where the 'val' is the thing you want to send to callback. The list of int 'value' is a list of callback index.

Let's say you audio loop performs with vector size of 512 samples, you will be able to pass your callbacks after every 512 audio samples are processed, which means 48000 / 512 times per second (assuming you sample rate is 48000). This method is not the best one but it works, I still have to see if it works in very intensive processing context though. Here, it has been thought for realtime audio, but it could work the same for audio rendering.

You can see the full code here : https://framagit.org/johannphilippe/csounddart/-/blob/master/lib/csoundnative.dart

Purusha