Cannot run gRPC protoc in alpine based dotnet SDK

Question

See this github issue: https://github.com/grpc/grpc/issues/18338

See this example repo: https://github.com/slolife/alpine-protoc

If I include the Grpc.Tools 1.19.0 nuget package in a project, which adds a build step <Protobuf Include="Test.proto" />

This works fine if I create a docker image to build and use the microsoft/dotnet:2.2-sdk as the build image. However, if I try to use the alpine based microsoft/dotnet:2.2-sdk-alpine build image, the build fails with the following error message:

/root/.nuget/packages/grpc.tools/1.19.0/build/_protobuf/Google.Protobuf.Tools.targets(263,5): error MSB6003: The specified task executable "/root/.nuget/packages/grpc.tools/1.19.0/tools/linux_x64/protoc" could not be run. No such file or directory [/src/alpine-protoc.csproj]

I confirmed that the protoc file is in the location that the error message is complaining about.

I tried running apk add libc6-compat and re-running the build. This time I got the following error:

/root/.nuget/packages/grpc.tools/1.19.0/build/_protobuf/Google.Protobuf.Tools.targets(263,5): error MSB6006 : "/root/.nuget/packages/grpc.tools/1.19.0/tools/linux_x64/protoc" exited with code 139. [/src/alpine-proto c.csproj]

---

Update: Output from ldd protoc

~/.nuget/packages/grpc.tools/1.19.0/tools/linux_x64 # ldd protoc
/lib64/ld-linux-x86-64.so.2 (0x7f60935a7000)
libm.so.6 => /lib64/ld-linux-x86-64.so.2 (0x7f60935a7000)
libpthread.so.0 => /lib64/ld-linux-x86-64.so.2 (0x7f60935a7000)
libc.so.6 => /lib64/ld-linux-x86-64.so.2 (0x7f60935a7000)

Sible

May be due to dynamic linking failure. Could you paste the output of ldd <protoc executable>? – Tumescent

@Tumescent thanks, updated the question with that output – Sible

Answer 1

This seems like a libc compatibility issue - my best guess is that dotnet pulls the mainland, glibc compatible protoc along with grpc.

The No such file or directory error on Alpine when running an executable that is present and accessible, is typical to ld failing to resolve dependant libraries such as libc.so.6.

From ldd protoc output we can see that protoc requires libc.so.6, so it was likely built on Linux with glibc, such as Ubuntu or Debian. The libc6-compat package provides a compatibility layer on top of musl libc, to allow basic glibc functionality, for example, adding the required library files and missing functions. However, it does not provide full glibc compatiblity. Complex applications depending on glibc are not likely to work out of the box when linked against musl libc, at least not without some porting effort.

When you added libc6-compat, protoc was able to link against the musl-glibc compatiblity libraries, libc.so.6 et al, but when run exited with code 139, meaning that it segfaulted (got a SIGSEGV). This is a good indication that you must use it with an actual glibc. One possible reason for this is the default stack size: musl libc creates threads with very small default stack size, around 68kb, whereas glibc threads are created with 2-8MB stacks. For other subtle differences, refer to: https://wiki.musl-libc.org/functional-differences-from-glibc.html.

You could try work around the nuget package incompatility using a simple hack: install the Alpine compatible protobuf compiler, with apk add protobuf; then, replace your protoc with a symlink to /usr/bin/protoc.

Alternatively, you could try installing proper glibc on Alpine, by adding the following to your Dockerfile (thanks to sgerrand and anapsix):

ENV GLIBC_REPO=https://github.com/sgerrand/alpine-pkg-glibc
ENV GLIBC_VERSION=2.28-r0

RUN set -ex && \
    apk --update add libstdc++ curl ca-certificates && \
    for pkg in glibc-${GLIBC_VERSION} glibc-bin-${GLIBC_VERSION}; \
        do curl -sSL ${GLIBC_REPO}/releases/download/${GLIBC_VERSION}/${pkg}.apk -o /tmp/${pkg}.apk; done && \
    apk add --allow-untrusted /tmp/*.apk && \
    rm -v /tmp/*.apk && \
    /usr/glibc-compat/sbin/ldconfig /lib /usr/glibc-compat/lib

Tumescent

I went with the symlink route to test it and that was the ticket. Needed both protoc and libc6-compat to get the whole thing working. Posted the final changes to the github.com/slolife/alpine-protoc repo. Hopefully the Grpc.Tools package will be updated to support out of the box – Sible

Grpc.Tools also allows overriding the protoc path via the environment variable PROTOBUF_PROTOC. – Connieconniption

Answer 2

I got this to work by adding the following to my Dockerfile:

ENV PROTOBUF_PROTOC=/usr/bin/protoc
ENV gRPC_PluginFullPath=/usr/bin/grpc_csharp_plugin
RUN apk add protobuf protobuf-dev grpc

---

UPDATE 2024: Starting from alpine v3.17, the grpc package no longer provides plug-ins. You need to install the grpc-plugins package independently. Like this:

ENV PROTOBUF_PROTOC=/usr/bin/protoc
ENV gRPC_PluginFullPath=/usr/bin/grpc_csharp_plugin
RUN apk add protobuf protobuf-dev grpc grpc-plugins

Meara

Worked for my as well. That's the way to go. Thank you @Meara – Hornstone

💯 GH issues are outdated, this is the good one – Phosgenite

Answer 3

I ran into this problem on a Raspberry Pi. This script installs the proto compiler from the raspbian repo, and sets up symbolic links.

#!/bin/bash

# Get the first occurrence of protoc in PATH directories
protoc_path=$(which protoc)


# Check if protoc exists in PATH directories
if [ -z "$protoc_path" ]; then
    echo "protoc not found in PATH. Installing protobuf-compiler..."
    sudo apt update && sudo apt install -y protobuf-compiler protobuf-compiler-grpc

    # Recheck for protoc
    protoc_path=$(which protoc)
    if [ -z "$protoc_path" ]; then
        echo "Failed to install protobuf-compiler or protoc not found in PATH"
        exit 1
    fi
fi

proto_plugin_path=$(which grpc_csharp_plugin)

for dir in /root/.nuget/packages/grpc.tools/*/tools/linux_arm64; do
    if [ -d "$dir" ]; then
        if [ ! -L "$dir/protoc" ]; then
          rm -f "$dir/protoc" || true
          ln -s "$protoc_path" "$dir/protoc"
        fi
        
        if [ ! -L "$dir/grpc_csharp_plugin" ]; then
          rm -f "$dir/grpc_csharp_plugin"
          ln -s "$proto_plugin_path" "$dir/grpc_csharp_plugin"
        fi
    fi
done

export PROTOBUF_TOOLS_OS=linux
export PROTOBUF_TOOLS_CPU=arm64
dotnet run 

Dishtowel