Proper way of getting the address of non-exported kernel symbols in a Linux kernel module

Question

I'm currently working on a Linux kernel module to intercept some syscalls for printing statistics about them system-wide.

I've come across different ways of getting the address of the sys_call_table symbol, but have yet to find a way that works on recent kernels (e.g. 5.11). On older kernels, wouldn't we have used kallsyms_lookup_name? It looks like that symbol is no longer exported.

I could just look at /proc/kallsyms, but this seems like a bad idea and not generalizable. What are other alternatives?

Rhinoceros

"I could just look at /proc/kallsyms but this seems like a bad idea and not generalizable." - FYI, replacing a system calls is a bad pattern too. It is used by some beginners as a "task" for their first kernel module. But Linux kernel is never encourages replacing a system calls. – Favour

Answer 1

Disclaimer: using non-exported symbols is in general not a good idea, so you should only do it for testing/educational purposes, not for production-ready modules/drivers.

Before Linux v5.7, you indeed would have used kallsyms_lookup_name() to look-up non-exported kernel symbols from a module. See How do I access any kernel symbol in a kernel module? if you want to know how.

However, the symbol stopped being exported in v5.7 because nobody was using it outside of core kernel code, and it was just there to be abused by modules to find and use other non-exported symbols. Here's also a relevant LWN article on this. Nowadays there isn't really a "proper way" to work around this problem, but there a number of different "hacks" that you could consider.

The following approaches cover both kernel functions and global objects (i.e. global variables):

1. If you are already compiling the kernel, you can add EXPORT_SYMBOL() after the definition of the symbol(s) you are interested in. This is the simplest option given you are willing to modify the kernel and build a custom one. You could also export kallsyms_lookup_name() in kernel/kallsyms.c and then use that, if you really want.

2. You can use an unsigned long module parameter, passing the needed symbol address (taken from /proc/kallsyms) when loading the module, and then cast it to the appropriate type:

   static unsigned long addr;
   module_param_named(addr, addr, ulong, 0);
   MODULE_PARM_DESC(addr, "Address of the `foo` symbol");
   
   static <type_of_foo_here> *foo_ptr;
   // Examples:
   // int foo(char *)   -> int (*foo_ptr)(char *)
   // unsigned long foo -> unsigned long *foo_ptr
   
   static int __init mymodule_init(void)
   {
       foo_ptr = (typeof(foo_ptr))addr;
       // ...
       return 0;
   }
   

   Then you'd be able to do something like this:

   sudo insmod mymodule.ko addr=0x$(sudo grep ' some_symbol_name' /proc/kallsyms | cut -d' ' -f1)
   

3. If your kernel supports kprobes, you can [ab]use a kprobe to make the kernel lookup a symbol for you through kprobe_register(). This approach is detailed in this other answer. Due to the intended usage of kprobes, this will only work for functions, however you can simply find kallsyms_lookup_name() first, and then use that to lookup any other symbol.

   In order for this to work, your kernel needs to be configured with CONFIG_KPROBES=y as well as CONFIG_KALLSYMS=y (and possibly also CONFIG_KALLSYMS_ALL=y depending on the symbol you want), since register_kprobe() uses exactly kallsyms_lookup_name() under the hood. Automatic symbol address resolution for kprobes has been supported since Linux v2.6.16.

4. For functions only, you can also consider re-implementing the functionality in your module. For example, task_statm() implemented in fs/proc/task_mmu.c is a rather small function that only uses other exported functions, so "borrowing" it for use in your module would be rather straightforward.

   Chances are that you want to call some non-exported function for a more specific purpose than what it was designed for. In such case, a good idea would be to look at the kernel source to understand how it works, and only re-implement the bare minimum needed for your module.

5. Finally, you could technically open and read /proc/kallsyms from kernel space using filp_open() + kernel_read() from <linux/fs.h>, though this would probably be the objectively worst solution overall.

Eleonoreeleoptene

Answer 2

We could also find the address of the kallsyms_lookup_name function using kprobes.

Quotes taken from here (kprobes)

Kprobes enables you to dynamically break into any kernel routine and collect debugging and performance information non-disruptively. You can trap at almost any kernel code address

To register a kprobe, first a kprobe struct needs to be initialized with the name of the symbol that needs to be trapped. We can do that by setting the symbol_name in the kprobe struct.

#include <linux/kprobes.h>
static struct kprobe kp = {
    .symbol_name = "kallsyms_lookup_name"
};

The kprobe struct has the following elements within it (shortened for brevity):

struct kprobe {
    ...
    /* location of the probe point */
    kprobe_opcode_t *addr;

    /* Allow user to indicate symbol name of the probe point */
    const char *symbol_name;
    ...
}

With the introduction of the “symbol_name” field to struct kprobe, the probepoint address resolution will now be taken care of by the kernel.

Once the symbol_name is set, the address of the probe point is determined by the kernel. So, now all that's left to do is to register the probe, extract the probepoint address and then unregister it:

typedef unsigned long (*kallsyms_lookup_name_t)(const char *name);
kallsyms_lookup_name_t kallsyms_lookup_name;
register_kprobe(&kp);
kallsyms_lookup_name = (kallsyms_lookup_name_t) kp.addr;
unregister_kprobe(&kp);

We now have the kallsyms_lookup_name address. Using that we can find the sys_call_table address the old-fashioned way:

kallsyms_lookup_name("sys_call_table");

Source for kprobe struct

Source for kprobe technique

Laurellaurella

Is this all userspace and does not require the kernel to be recompiled? – Mitsue