SystemTap seems to give unrelevant output

Question

My system is Ubuntu, uname -r = 4.15.0-23-generic. I've installed debug symbols for a kernel.

My problem is:

---

I am trying to use socket(AF_PACKET, SOCK_DGRAM, 0) for transmision purpose. For sendto(fd, 0,0,0,0,0) syscall I've got EINVAL (Invalid Argument) and I am trying to investigate what the cause is.

---

Therefore, to find out what returns EINVAL I am using the SystemTap. The script below tracks execution statement by statement of tpacket_snd function.

my probe program: info.stp

probe kernel.statement("tpacket_snd@*:*") {
    tokenize(pp(),"@");
    printf("HIT %s\n", tokenize("","@"))
}

And here is the output of sudo stap info.stp for my implementation of that kind of transmission:

HIT /build/linux-uT8zSN/linux-4.15.0/net/packet/af_packet.c:2619")
HIT /build/linux-uT8zSN/linux-4.15.0/net/packet/af_packet.c:2627")
HIT /build/linux-uT8zSN/linux-4.15.0/net/packet/af_packet.c:2628")
HIT /build/linux-uT8zSN/linux-4.15.0/net/packet/af_packet.c:2636")
HIT /build/linux-uT8zSN/linux-4.15.0/net/packet/af_packet.c:2638")
HIT /build/linux-uT8zSN/linux-4.15.0/net/packet/af_packet.c:2640")
HIT /build/linux-uT8zSN/linux-4.15.0/net/packet/af_packet.c:2641")
HIT /build/linux-uT8zSN/linux-4.15.0/net/packet/af_packet.c:2656")
HIT /build/linux-uT8zSN/linux-4.15.0/net/packet/af_packet.c:2659")
HIT /build/linux-uT8zSN/linux-4.15.0/net/packet/af_packet.c:2658")
HIT /build/linux-uT8zSN/linux-4.15.0/net/packet/af_packet.c:2662")
HIT /build/linux-uT8zSN/linux-4.15.0/net/packet/af_packet.c:2663")
HIT /build/linux-uT8zSN/linux-4.15.0/net/packet/af_packet.c:2669")
HIT /build/linux-uT8zSN/linux-4.15.0/net/packet/af_packet.c:2671")
HIT /build/linux-uT8zSN/linux-4.15.0/net/packet/af_packet.c:2674")
HIT /build/linux-uT8zSN/linux-4.15.0/net/packet/af_packet.c:2672")
HIT /build/linux-uT8zSN/linux-4.15.0/net/packet/af_packet.c:2675")
HIT /build/linux-uT8zSN/linux-4.15.0/net/packet/af_packet.c:2680")
HIT /build/linux-uT8zSN/linux-4.15.0/net/packet/af_packet.c:2688")
HIT /build/linux-uT8zSN/linux-4.15.0/net/packet/af_packet.c:2692")
HIT /build/linux-uT8zSN/linux-4.15.0/net/packet/af_packet.c:2694")
HIT /build/linux-uT8zSN/linux-4.15.0/net/packet/af_packet.c:2693")
HIT /build/linux-uT8zSN/linux-4.15.0/net/packet/af_packet.c:2706")
HIT /build/linux-uT8zSN/linux-4.15.0/net/packet/af_packet.c:2710")
HIT /build/linux-uT8zSN/linux-4.15.0/net/packet/af_packet.c:2707")
HIT /build/linux-uT8zSN/linux-4.15.0/net/packet/af_packet.c:2708")
HIT /build/linux-uT8zSN/linux-4.15.0/net/packet/af_packet.c:2709")
HIT /build/linux-uT8zSN/linux-4.15.0/net/packet/af_packet.c:2712")
HIT /build/linux-uT8zSN/linux-4.15.0/net/packet/af_packet.c:2743")
HIT /build/linux-uT8zSN/linux-4.15.0/net/packet/af_packet.c:2728")
HIT /build/linux-uT8zSN/linux-4.15.0/net/packet/af_packet.c:2736")
HIT /build/linux-uT8zSN/linux-4.15.0/net/packet/af_packet.c:2735")
HIT /build/linux-uT8zSN/linux-4.15.0/net/packet/af_packet.c:2785")
HIT /build/linux-uT8zSN/linux-4.15.0/net/packet/af_packet.c:2787")
HIT /build/linux-uT8zSN/linux-4.15.0/net/packet/af_packet.c:2789")

[https://elixir.bootlin.com/linux/v4.15/source/net/packet/af_packet.c#L2618]

---

My question is: That output is unrelevant (doesn't match to) with sourcecode, because:

1. Firstly, it is pointed that line af_packet.c:2707 was executed after 2710 2710 contains no jump instruction.

---

2. Secondly, From my investigation I could conlude that condition: [lines 2741-2745]

   if (po->has_vnet_hdr && virtio_net_hdr_to_skb(skb, vnet_hdr, vio_le())) { tp_len = -EINVAL; goto tpacket_error; }

   was evaluated to the true- note that SystemTap points that line 2743 was executed. But, from the other side I've investigated with SystemTap that po->has_vnet_hdr is equal to 0 so it is not possible to execute if body. However, SystemTap points it.

My question is:

How to repair it or what do I wrongly?

Syntactics

My initial reaction is that this is just compiler output with optimizations. There is no guarantee that the underlying assembler ordering will match the source code. – Presentable

For SOCK_DGRAM sockets, you must specify a destination address at every sendto(). Passing NULL, 0 for addr, addr_len is invalid. – Approve

Answer 1

Q1: Peter is quite right. After compiler optimizations, the source lines may appear to be executed in non-linear sequence, as the instructions of different C statements are intermingled. This does not represent a problem.

Q2: To figure out which statement caused the -EINVAL return, I would combine statement probes (like you have, except also printing $$vars to see the local variables) AND a function .call/.return probe pair, to note when the function is being left. The last few statement trace lines before the return would be where I'd look for the cause.

Shrift