Let's see...
Loading
Loading a program consists in injecting its instructions into the kernel via the bpf(BPF_PROG_LOAD, ...) system call (for most program types). The program passes the verifier, which runs a number of checks and may rewrite some instructions (in particular for map access). Then the program may be JIT-compiled, if JIT-compiling is enabled. The program as defined in the kernel memory is a struct bpf_prog object containing (or pointing to) information about the program, including its eBPF bytecode and (if relevant) JIT-compiled instructions.
At the end of this process, the program is located in the kernel memory. It is not attached to a particular object. It has a reference counter, and the kernel keeps it until the counter reaches zero. References can be held by file descriptors to the program: one is returned by the bpf() system call to the loading application, for example. Other references can be created by attaching, linking, pinning the program, or (if I remember correctly) referencing it in a prog_array map. If no reference remains (for example, the loading application exits right after loading the program, thus closing its file descriptor pointing to the program), then it is removed from the kernel.
The notion of “attach type” depends on the program type. Some program types don't have this notion: XDP programs just attach to the XDP hooks for an interface. Programs attached to cgroups do have an “attach type” that tells where, exactly, the program is attached.
Loading the program is mostly separate from these attach types. However, some program types - not all - do require the user to pass the expected attach type at load time, through the expected_attach_type field of the union bpf_attr object passed to the bpf() system call. This expected attach type is used by the verifier and the syscall handler to perform various verifications.
Attaching
Your understanding of the attachment step sounds good. Based on its attach and/or program type, the program is “attached” to the hook where it is supposed to run. The relevant kernel structures, cgroup_bpf->effective in your case, will point to the program (not store it - the program does not move, cgroup_bpf->effective simply points to a list of stuct bpf_prog *), and the events occurring at this hook will trigger the program.
Note that for some program types, such as networking or cgroup-attached programs, attaching the program increases its reference counter, so that the loading application can exit without the program being removed from the kernel. For some other program types, such as kprobes, this is not sufficient to keep the program open, because attaching is based on a file descriptor returned by perf_event_open() to keep the program attached, and a process needs to keep running to hold this file descriptor open.
Linking
How do we keep eBPF probes running when the loading application closes? This is where eBPF links come into play. eBPF programs can be attached to links instead of their traditional hook. The links themselves are attached to the kernel hooks. This offers a better interface for manipulating programs. One advantage is that it is possible to pin such links, to keep eBPF probes running when their loading application exits. Another advantage is to make it easier to keep track of the references held on a program, and to make sure that no eBPF program remains loaded if the loading application exits unexpectedly.
Are links “special types of attachment”? I'm not sure exactly. Looking at the code, it seems that the tracing hooks now always work with links on newer kernels. For other program types, the interface provided by eBPF links was added later, and seems to live beside the traditional hooks. For cgroups for example, you can attach programs the old way (going through cgroup_bpf_prog_attach(), or you can load them, create an eBPF link and attach your program to that link (through link_create()) - as you observed, in both case you will end up running cgroup_bpf_attach().
I do not believe there is good documentation about eBPF links at the moment, so the best we have is probably the cover letters and commit logs from the patch sets:
eBPF links are not to be confused with linking of the ELF object files used to store the bytecode before it is loaded into the kernel. For example, libbpf is able to link several object files containing various eBPF functions or subprograms or other objects, and to produce a single output ELF file containing all those objects. This is not related to the “bpf_link” interface.
Pinning
Pinning is a way to hold references to eBPF objects (programs, but also maps, or links). It is done with the bpf(BPF_OBJ_PIN, ...) system call, it creates a path in the eBPF virtual file system, and a file descriptor to the object can later be retrieved by open()-ing that path. As long as an object is pinned, it remains in the kernel. Pinning a program or a map is not necessary to run it. As long as other references exist (file descriptors, or the program is attached to certain hooks; or for maps, they are referenced by existing programs...), the program remains loaded in kernel memory and, if attached, it can run.
In particular, pinning an eBPF link ensures that the program attached to that link remains loaded after its loading application terminates and closes its file descriptors.
Summary
- Loading: Injection of the program into the kernel, verifier kicks in, it may rewrite some instructions and link to relevant internal eBPF objects (BTF, maps, etc.), and JIT-compilation may occur. The field
expected_attach_type may be necessary.
- Attaching: The program is attached to the hook related to its program type, using the provided attach type if relevant.
- Linking: Depending on program types or if required, the program is attached to an eBPF link instead of directly to its regular attach point. The link is attached to the regular hook, and provides a more flexible interface to manage the program.
- Pinning: Programs or links (or maps) can be pinned to the bpffs to make them persistent (but not across reboot).
