valgrind returning an unhandled instruction on Raspberry Pi

Question

I've been trying to debug a segmentation fault recently using valgrind on my raspberry Pi (model b), running Debian GNU/Linux7.0 (wheezy). Every time I run valgrind on a compiled C++ program, I get something like the following:

disInstr(arm): unhandled instruction: 0xF1010200
    cond=15(0xF) 27:20=16(0x10) 4:4=0 3:0=0(0x0)
valgrind: Unrecognized instruction at address 0x4843638.
at 0x4843638: ??? (in /usr/lib/arm-linux-gnueabihf/libconfi_rpi.so)

Then the normal valgrind stuff, causing a SIGILL and terminating my program. At first I assumed there was some memory leak in my program that was causing it to execute a piece of non-instruction memory as an instruction, but then I ran the following hello world code, and got the same result.

#include <iostream>
using namespace std;

int main() {
cout<<"Hello World"<<endl;

return 0;
}

There can't possibly be a memory leak/segfault in that, so why is it giving me this error? I'm pretty new with valgrind, but I ran it with the most basic valgrind ./a.out.

Flemings

try this: valgrind --tool=memcheck --leak-check=yes --show-reachable=yes --num-callers=20 --track-fds=yes ./a and post what happens. – Smalley

why dont you use the gcc debugger gdb ? It is easy to know the cause of a segfault with it! – Electrochemistry

@Phong: It's not segfault. Valgrind isn't sure how to handle the instruction. – Tractile

@cornstalks: I know the problem is about valgrind. But he said he was first trying to debug a segfault. – Electrochemistry

Answer 1

From your code (a simple hello world), It complain about an Unrecognized instruction at address 0x4843638. My guess is:

• Since valgrind need to intercept your malloc system call function (c standard library). This allow valgrind to check how many resource you did allocated/free which is used for memory leak detection (for example). If valgrind does not recognize your standard library environement (or the instruction language of your processor), It may does not behave as expecting, which would be the cause of your crash. You should check valgrind version and download the one fitted for your platform.

EDIT :

http://valgrind.org/docs/manual/faq.html

3.3. My program dies, printing a message like this along the way:

vex x86->IR: unhandled instruction bytes: 0x66 0xF 0x2E 0x5

One possibility is that your program has a bug and erroneously jumps to a non-code address, in which case you'll get a SIGILL signal. Memcheck may issue a warning just before this happens, but it might not if the jump happens to land in addressable memory.

Another possibility is that Valgrind does not handle the instruction. If you are using an older Valgrind, a newer version might handle the instruction. However, all instruction sets have some obscure, rarely used instructions. Also, on amd64 there are an almost limitless number of combinations of redundant instruction prefixes, many of them undocumented but accepted by CPUs. So Valgrind will still have decoding failures from time to time. If this happens, please file a bug report.

EDIT2 :

From wikipedia, the Raspberry Pi CPU:

• 700 MHz ARM1176JZF-S core (ARM11 family, ARMv6 instruction set)[3]

2.11. Limitations

On ARM, essentially the entire ARMv7-A instruction set is supported, in both ARM and Thumb mode. ThumbEE and Jazelle are not supported. NEON, VFPv3 and ARMv6 media support is fairly complete.

Your program/library just happened to have some instruction which is not supported yet.

Electrochemistry

Actually what you want to do is check for the newest release. If there is none, then this is a valgrind bug. Fill an issue and hope it should be fixed asap. (Valgrind project is big, so if they can reproduce it, it should not take long) – Electrochemistry

"Download the one fitted for your platform": I installed mine with apt-get, so it is specifically fitted for Raspbian. I will try latest version. – Tractile

I just check the instruction set compatibility between Raspberry(ARMv6) and valgrind. And it appear from the faq that ARMv6 is not completely supported by valgrind. (I updated the answer according to it). Sorry to notice this late. I fear you cant use the actuall valgrind on your Raspberry... – Electrochemistry

Answer 2

TL;DR: remove the package raspi-copies-and-fills if you're using Raspbian. It may also work in some other Linux variations like NOOBS.

---

As Phong already noted, this instruction is not supported by Valgrind. There is a bug report which explains the issue:

This keeps cropping up, for example most recently in bug 366464. Maybe I should explain more why this isn't supported. It's because we don't have a feasible way to do it. Valgrind's JIT instruments code blocks as they are first visited, and the endianness of the current blocks are "baked in" to the instrumentation. So there are two options:

(1) when a SETEND instruction is executed, throw away all the JITted code that Valgrind has created, and JIT new code blocks with the new endianness.

(2) JIT code blocks in an endian-agnostic way and have a runtime test for each memory access, to decide on whether to call a big or little
endian instrumentation helper function.

(1) gives zero performance overhead for code that doesn't use SETEND but a gigantic (completely infeasible) hit for code that does.

(2) makes endian changes free, but penalises all memory traffic regardless of whether SETEND is actually used.

So I don't find either of those acceptable. And I can't think of any other way to implement it.

In other words, it is hard to implement this instruction in valgrind. Summarizing the thread: the most common cause of this instruction are some faster memory management functions which the Raspberry Pi ships (memcmp, memset, etc.).

I solved it by (temporarily) removing raspi-copies-and-fills from my Raspbian install.

Schiffman

Answer 3

On Raspberry Pi 3 with NOOBS install of Raspian, implement ayke's answer by doing the following in a terminal window:

1. cd /etc
2. sudo nano ld.so.preload
3. Remove the line that includes "libarmmem.so" (/usr/lib/arm-linux-gnueabihf/libarmmem.so)
4. Save and exit ld.so.preload
5. Run valgrind
6. Place the line back into ld.so.preload after valgrind testing is complete

The pre-loaded "libarmmem.so" contains the instruction "setend" in the "memcmp" function which causes the unhandled instruction error. The standard library (used when the pre-loaded "libarmmem.so" library is not loaded) does not include the "setend" instruction in "memcmp".

Edgy

one line solution that worked for me and is easier to reverse: "sudo mv /etc/ld.so.preload /etc/ld.so.preload.disabled" – Schizoid

Answer 4

Valgrind is apparently problematic on Raspberry Pi:

https://web.archive.org/web/20131003042418/http://www.raspberrypisoft.com/tag/valgrind/

I suggest using other tools to find the seg fault.

Agateware

Answer 5

libarmmem.so's memcmp() uses multi-byte load instructions while searching for a difference in the buffers with a technique called SWAR. However, it used the 'setend be' instruction to put the processor into big-endian mode so that it was easier to locate the difference once the registers mismatched, then after the difference was located, 'setend le' to restore the endianness mode before returning. This worked great on ARM11, but...

Valgrind doesn't implement setend emulation for Arm processors(possibly because it would make the normal case of native-endian accesses slower), so it triggered the 'unrecognized instruction'.

SWAR-vectorized memcmp() can be implemented without SETEND, and I did so, and it's merged upstream. The performance impact is roughly negligible on large enought comparisons on the relevant cores.

@ayke is also correct, that disabling libarmmem will also work, but this will change out all of libarmmem, not just memcmp().

Kloof

I no longer see this problem (Jan 2024). There is a separate issue that Valgrind is not redirecting libarmmem functions for tools that need it (memcheck, dhat, drd, helgrind). – Phylloid