eBPF - Cannot read argv and envp from tracepoint sys_enter_execve

Question

I am learning BPF for my own fun, and I am having a hard time figuring out how to read argv and envp from the context passed to my eBPF program for sys_enter_execve

I will show my BPF program here and then explain in more details later what I am trying to accomplish.

Here's my code:

#include <linux/bpf.h>
#include <bpf_helpers.h>

struct
{
    __uint(type, BPF_MAP_TYPE_ARRAY);
    __type(key, __u32);
    __type(value, char[300]);
    __uint(max_entries, 1);
} mymap SEC(".maps");

// Based on /sys/kernel/debug/tracing/events/syscalls/sys_enter_execve/format
struct execve_args {
    short common_type;
    char common_flags;
    char common_preempt_count;
    int common_pid;
    int __syscall_nr;
    char *filename;
    const char *const *argv;
    const char *const *envp;
};

SEC("tracepoint/syscalls/sys_enter_execve")
int bpf_prog(struct execve_args *ctx) {
  
  __u32 index = 0;
  __u64 *value = bpf_map_lookup_elem(&mymap, &index);

  // An array of length 300 is purely arbitrary here
  char fn[300];

    // null check for the value fetched from the map
    if (value){
     
      // trying here to get the first env var passed to the process
      // started with execve
      const char *const first_env_value = ctx->envp[0];

      // null check
      if (!first_env_value){
        return 0;
      }
      
      // trying to safely read the value pointed by first_env_value
      bpf_probe_read_user_str(fn, sizeof(fn), first_env_value);
      bpf_map_update_elem(&mymap, &index, fn, BPF_ANY);
      return 0;
    
    }

    return 0;
}

char _license[] SEC("license") = "GPL";

What I want, here, is to ultimately read the first environments variable referenced by ctx->envp and save it in the map.

Building the program succeeds, but it fails when I try to load it into the kernel:

8: (15) if r0 == 0x0 goto pc+15
 R0_w=map_value(id=0,off=0,ks=4,vs=300,imm=0) R6_w=ctx(id=0,off=0,imm=0) R10=fp0 fp-8=mmmm????
; const char *const first_env_value = ctx->envp[0];
9: (79) r1 = *(u64 *)(r6 +32)
; const char *const first_env_value = ctx->envp[0];
10: (79) r3 = *(u64 *)(r1 +0)
R1 invalid mem access 'inv'
processed 10 insns (limit 1000000) max_states_per_insn 0 total_states 0 peak_states 0 mark_read 0

I use bpf2go from Cilium project to load the BPF program into the kernel. And I use a Go program to read there BPF map.

Can someone give me some hints as to what am I doing wrong?

Maybe it is the double pointer that confuses me (const char *const *envp), maybe I misunderstand the sys_enter_execve system call and the tracepoint inputs, etc.

Any hint would be appreciated!

I'm not a kernel developer. I mostly code in Go and Python, but I really want to learn how to write BPF programs in pure C, just for the fun of it.

Thanks in advance

Skellum

Answer 1

TL;DR. You are trying to read arbitrary kernel memory. You need to use bpf_probe_read for that.

---

Let's have a look at the error logs:

The invalid memory access is on a load from r1. The value in r1 was loaded from memory using the address in r6 as the base. According to the second line, the verifier associates type ctx to r6.

So r6 points to your variable ctx. That variable is special (hence why the verifier has a special ctx type for it). Your BPF program is allowed to access memory pointed by that variable as long as its bounded (the exact bound depends on the program type).

8: (15) if r0 == 0x0 goto pc+15
 R0_w=map_value(id=0,off=0,ks=4,vs=300,imm=0) R6_w=ctx(id=0,off=0,imm=0) R10=fp0 fp-8=mmmm????
; const char *const first_env_value = ctx->envp[0];
9: (79) r1 = *(u64 *)(r6 +32)
; const char *const first_env_value = ctx->envp[0];
10: (79) r3 = *(u64 *)(r1 +0)
R1 invalid mem access 'inv'

However, the value you retrieve from ctx->envp (the value stored in r1) is not part of ctx and may point to arbitrary kernel memory. The BPF verifier thus can't ensure ahead-of-time the safety of that access and rejects your program.

You need to use a BPF helper, bpf_probe_read, to access that memory. That helper will perform runtime checks to ensure the memory access is safe. If it's unsafe, it will return a negative error.

Salters

Thanks! That was exactly what I needed. I have published an answer below with your suggestion. It works now! I have marked your answer as the correct answer. – Skellum

Also thanks for taking the time to explain the error logs. I admit I did not looked too much into them as I did not understand them a lot. Now it makes more sens. I just need to remove the rust from my assembly language knowledge :) – Skellum

No problem, I'm glad it helped! The verifier's error logs can be a a bit intimidating at first, but it gets better with a little practice. It's mostly the same root causes all the time :-) – Salters

@pchaigno, However, the value you retrieve from ctx->envp (the value stored in r1) is not part of ctx I don't quite understand this part. If /sys/kernel/debug/tracing/events/syscalls/sys_enter_execve/format reports envp, why it is not considered a part of ctx? Thanks. – July

Because it is a pointer, not a value. envp is a value pointed by ctx – Skellum

Answer 2

Thanks a lot @pchaigno, you were absolutely right. To show other people how I solved my problem, here is the solution I have, based on pchaigno answer.

#include <linux/bpf.h>
#include <bpf_helpers.h>

struct
{
    __uint(type, BPF_MAP_TYPE_ARRAY);
    __type(key, __u32);
    __type(value, char[300]);
    __uint(max_entries, 1);
} mymap SEC(".maps");

// Based on /sys/kernel/debug/tracing/events/syscalls/sys_enter_execve/format
struct execve_args {
    short common_type;
    char common_flags;
    char common_preempt_count;
    int common_pid;
    int __syscall_nr;
    char *filename;
    const char *const *argv;
    const char *const *envp;
};

SEC("tracepoint/syscalls/sys_enter_execve")
int bpf_prog(struct execve_args *ctx) {
  
  __u32 index = 0;

  // Here we reserve a pointer to the first env var
  char *first_env_var;

  // Here we attempt to read the value pointed by ctx->envp[0] and store it in *first_env_var
  long res = bpf_probe_read(&first_env_var, sizeof(first_env_var), &ctx->envp[2]);

  // For demo purposes, simply return from the program 
  // if there is an error with bpf_probe_read
  if (res != 0){
    return 0;
  }
  
  // Read the value pointed by the (now) safe pointer *first_env_var
  // and store the value in 'value'
  char value[300];
  bpf_probe_read_str(value, sizeof(value), first_env_var);

  // Copy the value to the map
  bpf_map_update_elem(&mymap, &index, &value, BPF_ANY);
      
    return 0;
}

char _license[] SEC("license") = "GPL";

Skellum