passing function pointer to the C code using cgo

Question

Starting from Go v1.6 cgo changed the rules of passing pointers to the C code golang/go#12416. The example of invoking a dynamic Go callback from C code from the wiki doesn't work anymore.

package main

import (
    "fmt"
    "unsafe"
)

/*
   extern void go_callback_int(void* foo, int p1);

   // normally you will have to define function or variables
   // in another separate C file to avoid the multiple definition
   // errors, however, using "static inline" is a nice workaround
   // for simple functions like this one.
   static inline void CallMyFunction(void* pfoo) {
       go_callback_int(pfoo, 5);
       }
*/
import "C"

//export go_callback_int
func go_callback_int(pfoo unsafe.Pointer, p1 C.int) {
    foo := *(*func(C.int))(pfoo)
    foo(p1)
}

func MyCallback(x C.int) {
    fmt.Println("callback with", x)
}

// we store it in a global variable so that the garbage collector
// doesn't clean up the memory for any temporary variables created.
var MyCallbackFunc = MyCallback

func Example() {
    C.CallMyFunction(unsafe.Pointer(&MyCallbackFunc))
}

func main() {
    Example()
}

The output looks like this:

panic: runtime error: cgo argument has Go pointer to Go pointer

What is the proper way to do this today? Preferably without hacks like hiding pointer from the language by converting it into uintptr_t.

Shoe

Answer 1

Starting from Go 1.6 cgo has new rules.

Go code may pass a Go pointer to C provided that the Go memory to which it points does not contain any Go pointers.

[source]

These rules are checked during the runtime, and if violated program crashes. At the moment it is possible to disable checks using GODEBUG=cgocheck=0 environment variable. But in the future, that might stop working.

So it is not possible anymore to pass a pointer to C code, if the memory to which it is pointing stores a Go function/method pointer. There are several ways to overcome this limitations, but I guess in most of them you should store a synchronized data structure which represents the correspondence between a certain id and the actual pointer. This way you can pass an id to the C code, not a pointer.

The code solving this problem might look like this:

package gocallback

import (
    "fmt"
    "sync"
)

/*
extern void go_callback_int(int foo, int p1);

// normally you will have to define function or variables
// in another separate C file to avoid the multiple definition
// errors, however, using "static inline" is a nice workaround
// for simple functions like this one.
static inline void CallMyFunction(int foo) {
    go_callback_int(foo, 5);
}
*/
import "C"

//export go_callback_int
func go_callback_int(foo C.int, p1 C.int) {
    fn := lookup(int(foo))
    fn(p1)
}

func MyCallback(x C.int) {
    fmt.Println("callback with", x)
}

func Example() {
    i := register(MyCallback)
    C.CallMyFunction(C.int(i))
    unregister(i)
}

var mu sync.Mutex
var index int
var fns = make(map[int]func(C.int))

func register(fn func(C.int)) int {
    mu.Lock()
    defer mu.Unlock()
    index++
    for fns[index] != nil {
        index++
    }
    fns[index] = fn
    return index
}

func lookup(i int) func(C.int) {
    mu.Lock()
    defer mu.Unlock()
    return fns[i]
}

func unregister(i int) {
    mu.Lock()
    defer mu.Unlock()
    delete(fns, i)
}

This code comes from the (updated) wiki page.

Shoe

Answer 2

The solution has three parts.

1. Use a //export comment on the Go function to tell the cgo tool to produce a C wrapper for it (https://pkg.go.dev/cmd/cgo#hdr-C_references_to_Go).

2. Forward-declare that C identifier in the cgo C preamble, so that you can refer to the C wrapper within the Go code (https://golang.org/issue/19837).

3. Use a C typedef to convert that pointer to the correct C type, working around a cgo bug (https://golang.org/issue/19835).

Putting it all together:

package main

/*
static void invoke(void (*f)()) {
    f();
}

void go_print_hello();  // https://golang.org/issue/19837

typedef void (*closure)();  // https://golang.org/issue/19835
*/
import "C"

import "fmt"

//export go_print_hello
func go_print_hello() {
    fmt.Println("Hello, !")
}

func main() {
    C.invoke(C.closure(C.go_print_hello))
}

Trix

Answer 3

It depends exactly what you need to do with the callback function - but a trick that might work is to not pass the Go function, but a pointer to a struct with the function on it that you want.

For example:

package main

import (
    "fmt"
    "unsafe"
)

/*
extern void go_callback_int(void*, int);
static inline void CallMyFunction(void* pfoo) {
    go_callback_int(pfoo, 5);
}
*/
import "C"

//export go_callback_int
func go_callback_int(pfoo unsafe.Pointer, p1 C.int) {
    foo := (*Test)(pfoo)
    foo.cb(p1)
}

type Test struct {
}

func (s *Test) cb(x C.int) {
    fmt.Println("callback with", x)
}

func main() {
    data := &Test{}
    C.CallMyFunction(unsafe.Pointer(&data))
}

Another option might be to use a global variable as you had in your example, and then forgo passing anything useful in the pointer to C.

Like this:

package main

import (
    "fmt"
    "unsafe"
)

/*
extern void go_callback_int(void*, int);
static inline void CallMyFunction(void* pfoo) {
    go_callback_int(pfoo, 5);
}
*/
import "C"

//export go_callback_int
func go_callback_int(_ unsafe.Pointer, p1 C.int) {
    MyCallbackFunc(p1)
}

func MyCallback(x C.int) {
    fmt.Println("callback with", x)
}

// we store it in a global variable so that the garbage collector
// doesn't clean up the memory for any temporary variables created.
var MyCallbackFunc = MyCallback

func main() {
    C.CallMyFunction(nil)
}

It possibly depends on what your C callback needs to do.

If neither of these work it might be possible to do some tricks with channels to send the value you need (and keep it in scope).

Capacitate

As far as I understand what you're suggesting (not pass the Go function, but a pointer to a struct with the function on it that you want) won't work, since the new cgo rules state: Go code may pass a Go pointer to C provided that the Go memory to which it points does not contain any Go pointers. On top of that a runtime checks are used to enforce the rule. – Shoe

@IakovDavydov I tried the first example, and it worked for me, maybe go allowed this now? – Ganges

Answer 4

Another option since Go 1.17 is to use cgo/Handle

https://pkg.go.dev/runtime/[email protected]

https://golang.design/research/cgo-handle/

Wingover