Using Docker with nodejs with node-gyp dependencies

Question

I'm planning to use Docker to deploy a node.js app. The app has several dependencies that require node-gyp. Node-gyp builds these modules (e.g. canvas, lwip, qrcode) against compiled libraries on the delivery platform, and in my experience these builds can be highly dependent on the o/s version and libraries installed, and they often break a simple npm install.

So is building my Dockerfile FROM node:version the correct approach? This seems to be the approach shown in every Docker/Node tutorial I've found so far. But if I build from a node image, what will happen when I deploy the container? How can I ensure the target host will have the libraries needed to compile the node-gyp modules?

The other way I'm looking at is to build the Dockerfile FROM ubuntu:version. But I think this would mean installing nodeJS into the Ubuntu image and the whole thing would be much larger.

Are there other ways of handling this?

Willena

Answer 1

How can I ensure the target host will have the libraries needed to compile the node-gyp modules?

The target host is running docker as well. As long as the dependencies are in your image then your server has them as well. That's the entire point with docker if you ask me. If it runs locally, then it runs on the server as well.

I'd go with node-alpine (FROM node:8-alpine) for even smaller files. I struggled with node-gyp before I wrapped my head around it, but now I don't even see how I ever thought it was a problem. As long as you add build tools RUN apk add python make gcc g++ you are good to go (this adds some 100-200mb to the size however).

Also if it ever gets time consuming (say you find yourself rebuilding your image with --no-cache every now and then) then it can be a good idea to split it up into a base-image of your own and another image FROM my-base-image:latest which contains things that you change a more often.

There is some learning curve for sure, but I didn't find it that steep. At least not if you have touched docker before.

The other way I'm looking at is to build the Dockerfile FROM ubuntu:version.

I had only used CentOS before jumping on docker, and I run CentOS on my servers. So I thought it would be a good idea to run CentOS-images as well, but I found that to be just silly. There is absolutely zero gain unless you need something very OS-specific. Now I've only used alpine for maybe half a year, and so far the only alpine-specific command I've needed to learn is apk add/del.

And you probably know already, but don't spend too much time optimizing docker file size in the beginning. (You can reduce layer size a lot by combining commands on one line, (adding packages, running command, removing packages). But that cancels out the use of the docker image cache if you make any small changes in big layers. Better to leave that out until it matters.

Scarbrough

Thx ippi, it's starting to sink in--the docker node image is not just node, but has a set of libraries (plain, alpine, slim). I see now in the docker-node docs, the default image, "by design, has a large number of extremely common Debian packages. This reduces the number of packages that images that derive from it need to install, thus reducing the overall size of all images on your system." So I may have to load a lib or two, but probably not. ok, thx again. – Willena

I would use minideb instead of alpine; for node, use bitnami/node. alpine use musl libc which is incompatible with many current libraries. – Skate

You could also use multi stage builds to separate the build container from the runtime container. docs.docker.com/engine/userguide/eng-image/multistage-build – Macaw

The version of Alpine you use as a base is important! I've had all kinds of problems building form node:current-alpine (version 16 at the time of writing). Apparently, npm-16 isn't compatible with node-gyp. See #44316564 – Ghiberti

Answer 2

If you need to build stuff using node-gyp, you can add the line below, replacing your npm install or yarn install:

RUN apk add --no-cache --virtual .build-deps make gcc g++ python3 \
RUN npm install --production --silent \
RUN apk del .build-deps

Or even simpler, you can install alpine-sdk which is similar to Debian's build-essentials

RUN apk add --no-cache --virtual .build-deps alpine-sdk python3 \
RUN npm install --production --silent \
RUN apk del .build-deps

Source: https://github.com/mhart/alpine-node/issues/27#issuecomment-390187978

Kalle

You have to specify python2 or python3 in some cases otherwise you'll get an apk error – Swithbert

I recently downloaded Docker (July '22) and using the RUN \ RUN \... syntax was breaking. I needed to RUN <commands> \ && <commands> \ && <commands> - with only one RUN. – Mambo

To sum up the previous commenters. For me, it worked by replacing python with python3. I also replaced the last 2 RUN commands by &&, so they get executed together and the trailing `` has the right concatenating effect. – Simplehearted

Answer 3

Looking back (2 years later), managing node dependencies in a container is still a challenge. What I do now is:

1. Build the docker image FROM node:10.16.0-alpine (or other node version). These are official node images on hub.docker.com. Docker recommends alpine, and Nodejs builds on top of that, including node-gyp, so it's a good starting point;

2. Include a RUN apk add --no-cache to include all the libraries needed to build the dependent module, e.g. canvas (see example below);

3. Include a RUN npm install canvas in the docker build file; this builds the node module (e.g. canvas) into the docker image, so it gets loaded into any container run from that image.

But this can get ugly. Alpine uses different libraries from more heavy-weight OS's: notably, alpine uses musl in place of glibc. The dependent module may need to link to glibc, so then you would have to add it to the image. Sasha Gerrand offers one way to do it with alpine-pkg-glibc

Example installing node-canvas v2.5, which links to glibc:

#  geo_core layer
#  build on a node image, in turn built on alpine linux, Docker's official linux pulled from hub.docker.com

FROM node:10.16.0-alpine

#  add libraries needed to build canvas
RUN apk add --no-cache \
    build-base \
    g++ \
    libpng \
    libpng-dev \
    jpeg-dev \
    pango-dev \
    cairo-dev \
    giflib-dev \
    python \
    ; \

#  add glibc and install canvas
RUN apk --no-cache add ca-certificates wget  && \
    wget -q -O /etc/apk/keys/sgerrand.rsa.pub https://alpine-pkgs.sgerrand.com/sgerrand.rsa.pub && \
    wget https://github.com/sgerrand/alpine-pkg-glibc/releases/download/2.29-r0/glibc-2.29-r0.apk && \
    apk add glibc-2.29-r0.apk && \
    npm install [email protected]
    ;

Willena

See my comment below, but note that you can split out the compiler toolchain and -dev libraries into a builder base image, and then use a different base image for running that only includes the runtime libraries. (Cairo is a pig that will happily drag the entire x11 ecosystem with it!) – Camisado

Answer 4

2023 answer:

# Wont work with any version newer version of node
FROM --platform=linux/amd64 node:8-alpine

# Build dependencies
RUN apk add make gcc g++ python3

# Avoid "gyp ERR! stack Error: certificate has expired"
ENV NODE_TLS_REJECT_UNAUTHORIZED=0

WORKDIR /app

COPY . .

RUN npm install

EXPOSE 3000

CMD ["sh"]

Hake

# To use as a dev container, run: docker run -it -v $(pwd):/app -p 3000:3000 --entrypoint /bin/sh node:8-alpine – Hake

Answer 5

(In 2024) I like to build compatible pairs of "builder" and slim "runner" images. For Alpine/Node, here's a builder (your basket of dev libraries will vary, of course):

# node-alpine-builder
FROM node:20.15.1-alpine
RUN npm set registry "${NPM_REGISTRY}"
RUN apk update && apk --no-cache add bash curl file git jq build-base g++ make \
  python3 python3-dev cyrus-sasl-dev openssl-dev ca-certificates lz4-dev \
  musl-dev cairo-dev pango-dev giflib-dev librsvg-dev fontconfig
RUN npm install -g node-gyp

and here's a minimal runner (I usually add matching runtime libraries at the app Dockerfile level rather than in a kitchen sink base image):

# node-alpine-runner
FROM node:20.15.1-alpine
RUN apk update && apk --no-cache add openssl ca-certificates cyrus-sasl \
  lz4-libs curl file

(My Dockerfiles are expanded using envsubst.)

This then lets me make my project Dockerfile look like this:

FROM ${DOCKER_REGISTRY}/node-alpine-builder AS BUILDER
WORKDIR /build
COPY build/ .
RUN --mount=type=cache,target=/root/.npm npm ci --omit=dev --registry=${NPM_REGISTRY} && npm run build --if-present

FROM ${DOCKER_REGISTRY}/node-alpine-runner
WORKDIR /app
COPY . .
COPY --from=BUILDER /build/node_modules ./node_modules

HEALTHCHECK --start-period=120s CMD curl -fs http://localhost:8080/health | grep -q OK || exit 1

CMD ["node", "server.js"]

To seed the first phase, I copy just the package.json and package-lock.json files (and a .npmrc file with registry creds) into a temporary build directory.

The build is kicked off with:

docker buildx build --pull --push \
    --platform linux/arm64,linux/amd64 \
    -t "${DOCKER_REGISTRY}/${PACKAGE_NAME}:${VERSION}" \
    -t "${DOCKER_REGISTRY}/${PACKAGE_NAME}:${LATEST}" \
    .

(Note that if for whatever reason you can't or don't want to use multi-phase Dockerfiles, you can achieve much of the same toolchain-slimming effect by doing a docker run -v $(pwd)/build:/build -t node-alpine-builder npm ci and then copying the subsequent node_modules out into your slim image.)

Camisado