Error "no such instruction" while assembling project on Mac OS X

Question

I used homebrew to install GCC 4.7.0 and my project's make is failing at assembly-time. I can successfully take code from .c -> .s, but .s -> .o fails.

To view the brew formula used to install GCC, please look at: https://github.com/Homebrew/homebrew-dupes/blob/master/gcc.rb . I also installed binutils from upstream using https://github.com/mxcl/homebrew/blob/master/Library/Formula/binutils.rb . Install binutils does not appear to introduce a new 'as' in the /usr/local/lib or similar.

How can I track down this missing instruction error? It appears the compiler is correct but the linker simply does not support these instructions. The instructions do appear to be vector instructions.

xavierlange $> make suricata.o
/usr/local/bin/gcc-4.7 -DHAVE_CONFIG_H -I. -I..  -I../libhtp  -I/opt/local/include  -v -Wextra -Wall -fno-strict-aliasing -fno-tree-pre -Wno-unused-parameter -std=gnu99 -DOS_DARWIN -march=native   -DLIBPCAP_VERSION_MAJOR=1 -DHAVE_PCAP_SET_BUFF -DUNITTESTS -DREVISION="61d5fe3" -MT suricata.o -MD -MP -MF .deps/suricata.Tpo -c -o suricata.o suricata.c
Using built-in specs.
COLLECT_GCC=/usr/local/bin/gcc-4.7
Target: x86_64-apple-darwin11.3.0
Configured with: ../configure --enable-languages=c,c++,fortran,java,objc,obj-c++ --prefix=/usr/local/Cellar/gcc/4.7.0/gcc --datarootdir=/usr/local/Cellar/gcc/4.7.0/share --bindir=/usr/local/Cellar/gcc/4.7.0/bin --program-suffix=-4.7 --with-gmp=/usr/local/Cellar/gmp/5.0.4 --with-mpfr=/usr/local/Cellar/mpfr/3.1.0 --with-mpc=/usr/local/Cellar/libmpc/0.9 --with-system-zlib --enable-stage1-checking --enable-plugin --enable-lto --disable-multilib --disable-nls
Thread model: posix
gcc version 4.7.0 (GCC) 
COLLECT_GCC_OPTIONS='-mmacosx-version-min=10.7.3' '-D' 'HAVE_CONFIG_H' '-I' '.' '-I' '..' '-I' '../libhtp' '-I' '/opt/local/include' '-v' '-Wextra' '-Wall' '-fno-strict-aliasing' '-fno-tree-pre' '-Wno-unused-parameter' '-std=gnu99' '-D' 'OS_DARWIN' '-march=native' '-D' 'LIBPCAP_VERSION_MAJOR=1' '-D' 'HAVE_PCAP_SET_BUFF' '-D' 'UNITTESTS' '-D' 'REVISION=61d5fe3' '-MT' 'suricata.o' '-MD' '-MP' '-MF' '.deps/suricata.Tpo' '-c' '-o' 'suricata.o'
 /usr/local/Cellar/gcc/4.7.0/gcc/libexec/gcc/x86_64-apple-darwin11.3.0/4.7.0/cc1 -quiet -v -I . -I .. -I ../libhtp -I /opt/local/include -MD suricata.d -MF .deps/suricata.Tpo -MP -MT suricata.o -D__DYNAMIC__ -D HAVE_CONFIG_H -D OS_DARWIN -D LIBPCAP_VERSION_MAJOR=1 -D HAVE_PCAP_SET_BUFF -D UNITTESTS -D REVISION=61d5fe3 suricata.c -march=corei7-avx -mcx16 -msahf -mno-movbe -maes -mpclmul -mpopcnt -mno-abm -mno-lwp -mno-fma -mno-fma4 -mno-xop -mno-bmi -mno-bmi2 -mno-tbm -mavx -mno-avx2 -msse4.2 -msse4.1 -mno-lzcnt --param l1-cache-size=32 --param l1-cache-line-size=64 --param l2-cache-size=6144 -mtune=corei7-avx -fPIC -quiet -dumpbase suricata.c -mmacosx-version-min=10.7.3 -auxbase-strip suricata.o -Wextra -Wall -Wno-unused-parameter -std=gnu99 -version -fno-strict-aliasing -fno-tree-pre -o /var/folders/58/wg28y0x17p5c3s6y_2t2rj2h0000gn/T//ccKRmuNX.s
GNU C (GCC) version 4.7.0 (x86_64-apple-darwin11.3.0)
    compiled by GNU C version 4.7.0, GMP version 5.0.4, MPFR version 3.1.0-p8, MPC version 0.9
GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072
ignoring nonexistent directory "/usr/local/Cellar/gcc/4.7.0/gcc/lib/gcc/x86_64-apple-darwin11.3.0/4.7.0/../../../../x86_64-apple-darwin11.3.0/include"
ignoring nonexistent directory "/opt/local/include"
#include "..." search starts here:
#include <...> search starts here:
 .
 ..
 ../libhtp
 /usr/local/Cellar/gcc/4.7.0/gcc/lib/gcc/x86_64-apple-darwin11.3.0/4.7.0/include
 /usr/local/include
 /usr/local/Cellar/gcc/4.7.0/gcc/include
 /usr/local/Cellar/gcc/4.7.0/gcc/lib/gcc/x86_64-apple-darwin11.3.0/4.7.0/include-fixed
 /usr/include
 /System/Library/Frameworks
 /Library/Frameworks
End of search list.
GNU C (GCC) version 4.7.0 (x86_64-apple-darwin11.3.0)
    compiled by GNU C version 4.7.0, GMP version 5.0.4, MPFR version 3.1.0-p8, MPC version 0.9
GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072
Compiler executable checksum: 1dbaf0f0ba4d4d18b5d0d00c3c7d024d
COLLECT_GCC_OPTIONS='-mmacosx-version-min=10.7.3' '-D' 'HAVE_CONFIG_H' '-I' '.' '-I' '..' '-I' '../libhtp' '-I' '/opt/local/include' '-v' '-Wextra' '-Wall' '-fno-strict-aliasing' '-fno-tree-pre' '-Wno-unused-parameter' '-std=gnu99' '-D' 'OS_DARWIN' '-march=native' '-D' 'LIBPCAP_VERSION_MAJOR=1' '-D' 'HAVE_PCAP_SET_BUFF' '-D' 'UNITTESTS' '-D' 'REVISION=61d5fe3' '-MT' 'suricata.o' '-MD' '-MP' '-MF' '.deps/suricata.Tpo' '-c' '-o' 'suricata.o'
 as -arch x86_64 -force_cpusubtype_ALL -o suricata.o /var/folders/58/wg28y0x17p5c3s6y_2t2rj2h0000gn/T//ccKRmuNX.s
/var/folders/58/wg28y0x17p5c3s6y_2t2rj2h0000gn/T//ccKRmuNX.s:7791:no such instruction: `vcvtsi2ssq %rax, %xmm0,%xmm0'
/var/folders/58/wg28y0x17p5c3s6y_2t2rj2h0000gn/T//ccKRmuNX.s:7792:no such instruction: `vmovd %xmm0, %eax'
/var/folders/58/wg28y0x17p5c3s6y_2t2rj2h0000gn/T//ccKRmuNX.s:7799:no such instruction: `vcvtsi2ssq %rdx, %xmm0,%xmm0'
/var/folders/58/wg28y0x17p5c3s6y_2t2rj2h0000gn/T//ccKRmuNX.s:7800:no such instruction: `vaddss %xmm0, %xmm0,%xmm0'
/var/folders/58/wg28y0x17p5c3s6y_2t2rj2h0000gn/T//ccKRmuNX.s:7801:no such instruction: `vmovd %xmm0, %eax'
/var/folders/58/wg28y0x17p5c3s6y_2t2rj2h0000gn/T//ccKRmuNX.s:7804:no such instruction: `vmovd %eax, %xmm1'
/var/folders/58/wg28y0x17p5c3s6y_2t2rj2h0000gn/T//ccKRmuNX.s:7805:no such instruction: `vmovd %edx, %xmm2'
/var/folders/58/wg28y0x17p5c3s6y_2t2rj2h0000gn/T//ccKRmuNX.s:7806:no such instruction: `vdivss %xmm2, %xmm1,%xmm0'
/var/folders/58/wg28y0x17p5c3s6y_2t2rj2h0000gn/T//ccKRmuNX.s:7807:no such instruction: `vunpcklps %xmm0, %xmm0,%xmm0'
/var/folders/58/wg28y0x17p5c3s6y_2t2rj2h0000gn/T//ccKRmuNX.s:7808:no such instruction: `vcvtps2pd %xmm0, %xmm0'
/var/folders/58/wg28y0x17p5c3s6y_2t2rj2h0000gn/T//ccKRmuNX.s:7809:no such instruction: `vmovd %xmm0, %rax'
/var/folders/58/wg28y0x17p5c3s6y_2t2rj2h0000gn/T//ccKRmuNX.s:7819:no such instruction: `vmovd %rax, %xmm0'
make: *** [suricata.o] Error 1

Here is what happens when I use the GCC-4.7 installed by MacPorts (it uses cctools). I tried changing optimizations to see if that would help as well.

xavierlange $> /opt/local/bin/gcc-mp-4.7 -DHAVE_CONFIG_H -I. -I..  -I../libhtp  -I/usr/local/include -I/opt/local/include  -g -O0 -Wextra -Wall -fno-strict-aliasing -fno-tree-pre -Wno-unused-parameter -std=gnu99 -DOS_DARWIN -march=native   -DLIBPCAP_VERSION_MAJOR=1 -DHAVE_PCAP_SET_BUFF -DUNITTESTS -DREVISION="61d5fe3" -MT suricata.o -MD -MP -MF .deps/suricata.Tpo -S suricata.c -o suricata.s
xavierlange $> /opt/local/bin/as suricata.s suricata.s:9811:no such instruction: `vcvtsi2ssq %rax, %xmm0,%xmm0'
suricata.s:9812:no such instruction: `vmovd %xmm0, %eax'
suricata.s:9819:no such instruction: `vcvtsi2ssq %rdx, %xmm0,%xmm0'
suricata.s:9820:no such instruction: `vaddss %xmm0, %xmm0,%xmm0'
suricata.s:9821:no such instruction: `vmovd %xmm0, %eax'
suricata.s:9824:no such instruction: `vmovd %eax, %xmm1'
suricata.s:9825:no such instruction: `vmovd %edx, %xmm2'
suricata.s:9826:no such instruction: `vdivss %xmm2, %xmm1,%xmm0'
suricata.s:9827:no such instruction: `vunpcklps %xmm0, %xmm0,%xmm0'
suricata.s:9828:no such instruction: `vcvtps2pd %xmm0, %xmm0'
suricata.s:9829:no such instruction: `vmovd %xmm0, %rax'
suricata.s:9839:no such instruction: `vmovd %rax, %xmm0'

Fulgurate

The -force_cpusubtype_ALL option seems suspect. It specifies instructions that are common to all x86-64 CPUs. These are clearly AVX instructions, that aren't available prior to Sandy Bridge CPUs. – Taken

Fascinating. The computer in question is the 2011 MacBook Pro with Sandy Bridge. en.wikipedia.org/wiki/Advanced_Vector_Extensions says GCC 4.6 supports AVX. Even when I manually perform .s -> .o (no extra options on the as command) I still get the error. – Fulgurate

you might be running into issues when using the native assembler vs. the GNU assembler. link – Taken

Yes, that is why I tried installing binutils myself. The big issue is binutils (2.22) is not building "as" or "gas". Any tips on building "as" manually? – Fulgurate

Funny you should mention that... – Taken

I'd recommend setting up the MacPorts environment, and using their scripts to build gcc-4.7. I think this also builds the latest native cctools rather than using GNU as, ld, etc. – Taken

Ah, I knew nothing about cctools... fascinating as well! – Fulgurate

Ding ding ding, the logical conclusion to the immediate bug was to disable avx. CFLAGS="-mno-avx" is all I needed to move forward. This, however, does not solve the problem of why my compiler emits assembly I can't assemble. – Fulgurate

@BrettHale - if I am not mistaken, MacPorts does not supply AS or LD because upstream BinUtils does not supply it. Its there in source form, but its not enabled by Configure. Also see Please enable AS and LD in configure.ac for OS X. – Neutrino

Answer 1

The short solution would be removing '-march=native' option (at least it worked for me).

Lepage

Instead of falling back to the default SSE2, if you care about performance, you are better off with enabling optimization the instruction set closest to AVX which would be SSE4.2. So I'd rather replace -march=native with -msse4.2. – Intergrade

Even better: -march=native -mno-avx. Or see the answer by @capisce. – Clevelandclevenger

@Lepage Thank you very much for your powerful help. You deserve all best. You saved me a lot of time of search. Cheers. ab – Eohippus

Why downgrade and lose processor features? – Neutrino

Note this is not an option if you actually want/need AVX2 extensions (for example, in certain database systems or graphics processing). – Inrush

Answer 2

Adding -Wa,-q to the compiler flags fixed this problem for me. From the man pages for as:

-q

     Use the clang(1) integrated assembler instead of the GNU based system assembler.

The -Wa part passes it from the compiler driver to the assembler, much like -Wl passes arguments to the linker.

Chlortetracycline

When having includes (e.g. "-I/opt/local/include/") I get warnings of the type "clang: warning: argument unused during compilation: '-I /opt/local/include/'". Anyone else experiencing the same problem? I really like being able to build without any warnings. – Polydactyl

I tried this and got "clang: error: unsupported argument '-q' to option 'Wa,'" – Kelp

Perhaps this is a bit late, but that probably means you're using clang, which is aliased to gcc on OS X – Shipowner

Answer 3

It appears that you're running a version of as that doesn't know about the AVX extensions. You either need to build a newer gas or use clang to assemble.

Impostor

This does not work. I installed GCC from macports and tried going manually from .c -> .s and .s -> .o. I have updated the question with results. – Fulgurate

Perhaps I could suppress AVX instructions in the compiler? – Fulgurate