Dealing with Docker images in a Javascript/Typescript monorepo with unpublished packages

Question

I've been trying to find a clear answer to this problem, but unfortunately I haven't been able to. There are two similar SO questions that touch on this subject,

Should a developer be able to create a docker artifact from a lerna monorepo in their development environment?

How to build docker images in a lerna monorepo without publishing

but they don't answer it clearly, and I believe more context needs to be added to get a clearer answer.

I am setting up a hobby project using Next.js and several Express servers. I am building a set of microservices, and there will be a commons that can be shared among them. As this is a hobby project, having all of this exist in a single git repository makes sense. As well, being a hobby project, not having to publish any packages to a registry like NPM also makes sense.

Let's imagine a structure similar to what the lerna (a JS monorepo tool) docs showcase:

my-lerna-repo/
  package.json
  packages/
    package-1/
      package.json
    package-2/
      package.json

Using a tool like Lerna, you can have a commons package that is depended on, and have other packages reference it. Lerna will hoist up this commons package and simply symlink to it in the packages' node_modules.

This makes sense at development time, but how does this work in production when you are containerizing your services? For example, if we create a Dockerfile for each of our services and throw this in a Kubernetes cluster, the containers will fail as the commons dependency isn't there -- it was just a symlink.

I've looked into a "copy" argument for Lerna and there isn't one that exists. I am aware that there are other JS monorepo tools (such as Rush) but I know Lerna is the most popular.

How does one solve this problem of containerizing monorepo dependent packages without publishing them?

Rockafellow

Answer 1

For people who hit a similar issue in their development, I wanted to provide the solution I ended up with.

First, I setup a package.json in the root of my mono-repository using yarn workspaces:

{
  "name": "ticketing",
  "private": true,
  "workspaces": [
    "common",
    "service-one",
    "service-two"
  ]
}

Then, for each service's docker file, I would copy over the common projects:

FROM node:alpine as builder
WORKDIR /app

COPY package.json .
COPY common ./common
COPY service-one./service-one


FROM builder as prod
WORKDIR /app
ENV NODE_ENV=production

RUN npm install --prod
CMD npm run prod -w @project/service-one


FROM builder as dev
WORKDIR /app
ENV CI=true

RUN npm install
CMD npm run dev -w @project/service-one

With this setup, I didn't need to npm publish my commons anymore.

Rockafellow

This might work for 1-2 local packages but for larger project where packages themselves have references to other local packages this isn't viable solution – Colquitt

@AndrejK what alternative would you recommend for that case? – Rockafellow

@AnthonyA that solution might work but you will generate an image with all/src/** files which makes a really big-size image. I have a Dockerfile similar to the one you are using and using babel to transpile ES to commonJS in NodeJS and I only want to move the /dist folder and node_modules with symlinks folder to the final image for having a smaller deployable docker image – Fried