Can I have more than 32 netlink sockets in kernelspace?

Question

I have several kernel modules which need to interact with userspace. Hence, each module has a Netlink socket.

My problem is that these sockets interfere with each other. This is because all of them register to the same Netlink address family (because there aren't many available to begin with - the max is 32 and more than half are already reserved) and also because they all bind themselves to the same pid (the kernel pid - zero).

I wish there were more room for address families. Or, better yet, I wish I could bind my sockets to other pids. How come Netlink is the preferred user-kernel channel if only 32 sockets can be open at any one time?

libnl-3's documentation says

The netlink address (port) consists of a 32bit integer. Port 0 (zero) is reserved for the kernel and refers to the kernel side socket of each netlink protocol family. Other port numbers usually refer to user space owned sockets, although this is not enforced.

That last claim seems to be a lie right now. The kernel uses a constant as pid and doesn't export more versatile functions:

if (netlink_insert(sk, 0))
    goto out_sock_release;

I guess I can recompile the kernel and increase the address family limit. But these are kernel modules; I shouldn't have to do that.

Am I missing something?

Saddle

According to kernel sources, all kernel-side netlink sockets are stored in the single pre-allocated array (nl_table) and indexed by protocol number. So you unable to have more than 32(array size) netlink sockets. But nothing prevents you from using single socket and dispatching messages to it according to some predefined field in these messages. That's way you will have several "virtual" sockets. – Ardor

@Ardor in other words... I should code a special new module which would create a socket... and then multiplex that socket's messages across the other modules. You're right; it's one way to solve it, though it's a lot more trouble than it probably should. Perhaps this will end up being the accepted answer, though I'd rather wait to see more opinions for a while. – Saddle

Answer 1

No.

Netlink's socket count limit is why Generic Netlink exists.

Generic Netlink is a layer on top of stock Netlink. Instead of opening a socket, you register a callback on an already established socket, and listen to messages directed to a "sub"-family there. Given there are more available family slots (1023) and no ports, I'm assuming they felt a separation between families and ports was unnecessary at this layer.

To register a listener in kernelspace, use genl_register_family() or its siblings. In userspace, Generic Netlink can be used via libnl-3's API (though it's rather limited, but the code speaks a lot and is open).

Saddle

Answer 2

You are confused by MAX_LINKS variable name. It is not a "maxumum amount of links", it's a "maximum amount of families". The things you listed are netlink families or IOW netlink groups. There are indeed 32 families. Each family dedicated to serve some particular purpose. For example NETLINK_SELINUX is for SELinux notification and NETLINK_KOBJECT_UEVENT is for kobject notifications (these are what udev handles).

But there are no restrictions on number of sockets for each of the family.

When you call netlink_create it's checking your protocol number which in case of netlink socket is netlink family like NETLINK_SELINUX. Look at the code

static int netlink_create(struct net *net, struct socket *sock, int protocol,
                           int kern)
{
...
         if (protocol < 0 || protocol >= MAX_LINKS)
                 return -EPROTONOSUPPORT;
...

This is how your MAX_LINKS is using.

Later, when to actually create socket it invokes __netlink_create, which in turn calls sk_alloc, which in turn calls sk_prot_alloc. Now, in sk_prot_alloc it allocates socket by kmallocing (netlink doesn't have its own slab cache):

slab = prot->slab;
if (slab != NULL) {
    sk = kmem_cache_alloc(slab, priority & ~__GFP_ZERO);
    if (!sk)
        return sk;
    if (priority & __GFP_ZERO) {
        if (prot->clear_sk)
            prot->clear_sk(sk, prot->obj_size);
        else
            sk_prot_clear_nulls(sk, prot->obj_size);
    }
} else
    sk = kmalloc(prot->obj_size, priority);

Lisp

Hmm... sorry; I don't think you see my problem. Yes, I know the 32 refers to families. Ideally, I'd want to open more sockets per family. It's the fact that each family can have one pid (port) representative in kernelspace that stops me from doing so. I was thinking of using the families as a hack to open more sockets, but then I tie myself to the 32 max. But this is just a hack. – Saddle

What's important is that if I run netlink_kernel_create(net, NETLINK_USERSOCK, &cfg) once it binds itself to the usersock family's pid zero. If I do it a second time, it fails, presumably because pid zero of the usersock family is already taken. There doesn't seem to be any way to tell the function which pid I want to use. – Saddle