How to return a Hash/Raku object from native call?

Question

I am writing a library that uses NativeCall, it would be very convenient for me to be able to return a Raku Hash from an exported function. How can I do this?

For example, in Ruby, if I wanted to return a Hash from C, I would do something like:

#include "ruby.h"

VALUE make_hash() {
    VALUE hash = rb_hash_new();
    return hash;
}

I am interested to see if this can be done, I was thinking that maybe I would need to use a MoarVM header or something. But I'm not sure.

What I'm trying to do is write a C function that takes in a String does some stuff, then returns a Raku hash.

Garth

Does this help? (It's a wild guess. I don't understand what you're trying to do but no one else has posted anything and maybe something is better than nothing.) – Ratter

@Ratter Hmm it never occured to me to try function pointers, that might be the right way. Basically I need to take in a string (JSON), do some stuff in C with that string, and return a Hash. – Garth

Answer 1

I have done roughly this for Rust over here (this is a collection of some Raku-Rust Nativecall code examples, not a module)...

First the raku:

## Rust FFI Omnibus: Objects
## http:##jakegoulding.com/rust-ffi-omnibus/objects/

class ZipCodeDatabase is repr('CPointer') {
    sub zip_code_database_new() returns ZipCodeDatabase is native($n-path) { * }
    sub zip_code_database_free(ZipCodeDatabase)         is native($n-path) { * }
    sub zip_code_database_populate(ZipCodeDatabase)     is native($n-path) { * }
    sub zip_code_database_population_of(ZipCodeDatabase, Str is encoded('utf8'))
                                         returns uint32 is native($n-path) { * }

    method new {
        zip_code_database_new
    }

    submethod DESTROY {        # Free data when the object is garbage collected.
        zip_code_database_free(self);
    }

    method populate {
        zip_code_database_populate(self)
    }

    method population_of( Str \zip ) {
        zip_code_database_population_of(self, zip);
    }
}

my \database = ZipCodeDatabase.new;
database.populate;

my \pop1 = database.population_of('90210');
my \pop2 = database.population_of('20500');
say pop1 - pop2;

Then the Rust:

// Rust FFI Omnibus: Objects
// http://jakegoulding.com/rust-ffi-omnibus/objects/

pub struct ZipCodeDatabase {
    population: HashMap<String, u32>,
}

impl ZipCodeDatabase {
    fn new() -> ZipCodeDatabase {
        ZipCodeDatabase {
            population: HashMap::new(),
        }
    }

    fn populate(&mut self) {
        for i in 0..100_000 {
            let zip = format!("{:05}", i);
            self.population.insert(zip, i);
        }
    }

    fn population_of(&self, zip: &str) -> u32 {
        self.population.get(zip).cloned().unwrap_or(0)
    }
}

#[no_mangle]
pub extern "C" fn zip_code_database_new() -> *mut ZipCodeDatabase {
    Box::into_raw(Box::new(ZipCodeDatabase::new()))
}

#[no_mangle]
pub extern "C" fn zip_code_database_free(ptr: *mut ZipCodeDatabase) {
    if ptr.is_null() {
        return;
    }
    unsafe {
        Box::from_raw(ptr);
    }
}

#[no_mangle]
pub extern "C" fn zip_code_database_populate(ptr: *mut ZipCodeDatabase) {
    let database = unsafe {
        assert!(!ptr.is_null());
        &mut *ptr
    };
    database.populate();
}

#[no_mangle]
pub extern "C" fn zip_code_database_population_of(
    ptr: *const ZipCodeDatabase,
    zip: *const c_char,
) -> u32 {
    let database = unsafe {
        assert!(!ptr.is_null());
        &*ptr
    };
    let zip = unsafe {
        assert!(!zip.is_null());
        CStr::from_ptr(zip)
    };
    let zip_str = zip.to_str().unwrap();
    database.population_of(zip_str)
}

Obviously the C side of affairs will need to be quite different, but hopefully this gives enough clues.

Michael

This actually ended up being the approach I took, and it worked well. Thank you! – Garth

Answer 2

it would be very convenient for me to be able to return a Raku Hash from an exported function

A workaround could be to let the C function return a struct with key and values and then write a Raku wrapper that converts that into a Raku hash like this:

use v6;
use NativeCall;

constant LIB  = ('./libmylib.so');

class HInfo is repr('CStruct') is export {
    has Str $.key1;
    has num64 $.value1;
    has Str $.key2;
    has num64 $.value2;
}

sub foo_(Str--> HInfo) is native(LIB) is symbol('foo') { * }
sub foo(Str $str --> Hash) {
    my HInfo $hinfo = foo_($str);
    my %h;
    %h{$hinfo.key1} = $hinfo.value1;
    %h{$hinfo.key2} = $hinfo.value2;
    return %h;
}

my %h = foo("bar");
dd %h;

Darius

Hmm, I thought of this approach but this can cause issues because of nesting. If I need to nest a map inside of another map this doesn't work I don't think. I think I'm more leaning towards some sort of state that is modified via Raku callbacks passed to C. – Garth

@Ratter "Presumably one could just do the class HInfo is Hash is repr('CStruct') { ... } thing" I think the Hash type cannot be represented as CStruct. If I try this I get an error message: "CStruct representation only handles attributes of type: (u)int8, (u)int16, (u)int32, (u)int64, (u)long, (u)longlong, num32, num64, (s)size_t, bool, Str and types with representation: CArray, CPointer, CStruct, CPPStruct and CUnion" – Amphichroic

@RawleyFowler "If I need to nest a map inside of another map this doesn't work I don't think." I presume Håkon only meant their code to be a sketch of the possibilities with is repr. Do you mean you read something implying that the is repr approach won't work? Or you can't see how it could work conceptually? Or you tried it but ran into insurmountable problems? Or GH issues implied it won't work? Or you can see a way it should but there are missing pieces in is repr and native types that you think core devs won't have time to help you work through? Or something else? A combination? – Ratter

@Ratter After almost a week of deliberating it is really hard and really ugly to represent recursively nested data with CStruct, but it is possible. Mainly because you have to reinvent the wheel to bind them to your Raku code. For example, try to bind a std::map to Raku code. This is an ENORMOUS task, but it really shouldn't be, nested lists as-well can cause major issues, which left me having to use iterators over nested maps for my code. – Garth

Answer 3

As someone suggested, this is best done with a wrapper function. First things first though, what kind of value are you returning from C?

Your best bet is to return a CStruct.

Anatolia

I can't really encapsulate tree-like data with a CStruct though. – Garth