How To Set Up GitHub Actions to Publish a Lerna Monorepo

Question

I maintain a lerna/yarn monorepo. I'm in the process of migrating the CI/CD from circle to the new GitHuba Actions publish beta. I've created the following workflow:

name: CD

on:
  push:
    branches:
      - master

jobs:
  build:

    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@master

      - name: Checkout master
        run: git checkout master

      - name: Install rsync
        run: sudo apt install rsync

      - name: Install yarn
        run: |
          curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | sudo apt-key add -
          echo "deb https://dl.yarnpkg.com/debian/ stable main" | sudo tee /etc/apt/sources.list.d/yarn.list
          sudo apt-get update
          sudo apt-get install yarn

      - name: Install Packages
        run: yarn install

      - name: Test
        run: yarn test

      - name: Upload coverage results to Code Climate
        run: sh ./scripts/upload-coverage.sh
        env:
          CC_TEST_REPORTER_ID: ${{ secrets.CC_TEST_REPORTER_ID }}

      - name: Authenticate with Registry
        run: echo "registry=//registry.npmjs.org/:_authToken=$NPM_TOKEN" > ~/.npmrc
        env:
          NPM_TOKEN: ${{ secrets.NPM_TOKEN }}

      - name: Configure CI Git User
        run: |
          git config --global user.email [email protected]
          git config --global user.name GitHub Actions

      - name: Publish package
        run: yarn deploy --yes
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

      - name: Build Docs
        run: yarn docs

      - name: Deploy Docs
        run: |
          echo "apolloelements.dev" > docs/CNAME
          npx gh-pages --dist docs

It fails at the Publish Packages step with this message:

lerna info git Pushing tags...
lerna ERR! Error: Command failed: git push --follow-tags --no-verify origin master
lerna ERR! fatal: could not read Username for 'https://github.com': No such device or address
lerna ERR! 
lerna ERR!     at makeError (/home/runner/work/apollo-elements/apollo-elements/node_modules/execa/index.js:174:9)
lerna ERR!     at Promise.all.then.arr (/home/runner/work/apollo-elements/apollo-elements/node_modules/execa/index.js:278:16)
lerna ERR! Error: Command failed: git push --follow-tags --no-verify origin master
lerna ERR! fatal: could not read Username for 'https://github.com': No such device or address
lerna ERR! 
lerna ERR!     at makeError (/home/runner/work/apollo-elements/apollo-elements/node_modules/execa/index.js:174:9)
lerna ERR!     at Promise.all.then.arr (/home/runner/work/apollo-elements/apollo-elements/node_modules/execa/index.js:278:16)
lerna ERR! lerna Command failed: git push --follow-tags --no-verify origin master
lerna ERR! lerna fatal: could not read Username for 'https://github.com': No such device or address
lerna ERR! lerna 
error Command failed with exit code 128.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

Changing the remote to use HTTPS and the github token hasn't helped:

git remote rm origin
git remote add origin "https://$USER_NAME:[email protected]/apollo-elements/apollo-elements.git"

Where GITHUB_PERSONAL_ACCESS_TOKEN is a PAT passed via secrets.

In that case, I received this error:

lerna ERR! ENOREMOTEBRANCH Branch 'master' doesn't exist in remote 'origin'.

How should I set up the project to be able to push tags and commits back to the repository from CD?

Stretto

The developer.github.com/actions/managing-workflows/storing-secrets docs say "Every repository includes a GITHUB_TOKEN secret, but it's not available to an action by default. You must add the GITHUB_TOKEN secret to each action that requires access." I don't know what "adding the token secret to an action" entails, since I can't find details in the docs, and I'm not yet in the beta so I can't test that for myself. But if that involves some configuration step in the GitHub GUI, check that you've done that. – Expediency

Or perhaps, since "could not read Username" is showing up in the error you just need to edit your Lerna config to make the repo URLs be https://USERNAME:[email protected]/yourorg/yourrepo or something. It might be that simple. – Expediency

Thanks, I've edited the question to reflect the attempts I've made along those lines – Stretto

Ahh... now we're getting somewhere. We appear to have gotten connected to the repo, but we're getting a different error, probably because lerna's local copy doesn't know what branches the origin repo contains (because it's never done a fetch on it since you created the remote). So do a git fetch origin immediately after the git remote add origin $URL step, and then lerna's local copy of the repo should know what branches the origin repo contains, and should be able to push to origin's master` branch. – Expediency

Answer 1

UPDATED:

This configuration actually works end-to-end. The key features of this config are:

• setting the remote with git remote set-url origin https://$GITHUB_ACTOR:[email protected]/bennypowers/apollo-elements GITHUB_ACTOR is provided by the runner, GITHUB_PAT is a Github Personal Access Token set in the repository's secrets.
• re-checking and pulling with git checkout "${GITHUB_REF:11}" && git pull
• logging out of yarn, since lerna cannot handle yarn for whatever reason.
• using the specific, finicky .npmrc setup shown below, since this is a scoped package.
• running npm whoami after setting up auth. This will throw if authentication is broken lerna publish will push tags for each of your packages, and maybe write to the CHANGELOG.md and package.json files as well, even if it doesn't publish due to bad auth. Running npm whoami here to check that you actually can publish before running lerna prevents the headache of manually restoring the state of the repo.
• passing GITHUB_TOKEN, GH_TOKEN, and NPM_TOKEN to lerna publish

name: CD

on:
  push:
    branches:
      - master

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - name: checkout
        uses: actions/checkout@v1

      - name: Configure CI Git User
        run: |
          git config --global user.name '@bennypowers'
          git config --global user.email '[email protected]'
          git remote set-url origin https://$GITHUB_ACTOR:[email protected]/bennypowers/apollo-elements
        env:
          GITHUB_PAT: ${{ secrets.GITHUB_PAT }}

      - name: Checkout and pull branch
        run: git checkout "${GITHUB_REF:11}" && git pull

      - name: Install Packages
        run: yarn install

      - name: Authenticate with Registry
        run: |
          yarn logout
          echo "@apollo-elements:registry=http://registry.npmjs.org/" > .npmrc
          echo "registry=http://registry.npmjs.org/" >> .npmrc
          echo "//registry.npmjs.org/:_authToken=$NPM_TOKEN" >> .npmrc
          npm whoami
        env:
          NPM_TOKEN: ${{ secrets.NPM_TOKEN }}

      - name: Publish package
        run: lerna publish --yes --message 'chore: release new versions'
        env:
          GH_TOKEN: ${{ secrets.GITHUB_PAT }}
          GITHUB_TOKEN: ${{ secrets.GITHUB_PAT }}
          NPM_TOKEN: ${{ secrets.NPM_TOKEN }}

Note the above config redacts some irrelevant steps. See the complete workflow for the unredacted version

ORIGINAL ANSWER:

With help from StackOverflow user @rmunn, I arrived at this solution:

  - name: Configure CI Git User
    run: |
      git remote rm origin
      git remote add origin "https://$USER_NAME:[email protected]/apollo-elements/apollo-elements.git"
      git fetch
      git config --global user.email [email protected]
      git config --global user.name GitHub Actions
    env:
      USER_NAME: ${{ secrets.DEPLOYING_USER_NAME }}
      GITHUB_PAT: ${{ secrets.GITHUB_PAT }}

Where GITHUB_PAT is a personal access token with repo scope, saved in secrets.

The git fetch is required to set up local branches on the altered remote. The personal access token is required to push back to the repository.

Stretto

It's funny I'm looking at your repo, have mine setup the same way and am getting 'lerna ERR! ENOGIT Detached git HEAD, please checkout a branch to choose versions.' using this approach, will have a look and report back – Remediless

Ended up adding a checkout master step after the fetch, seemed to have fixed it – Remediless

Instead of using actions/checkout@v1 you should use: actions/checkout@v2 with fetch-depth: 0 and token: ${{ secrets.GITHUB_PAT}} – Unreserved

Could this be simplified now? docs.github.com/en/actions/language-and-framework-guides/… – Porringer

Is GITHUB_PAT an existing/known secret available in all Github repositories? When I try to create a repository secret called GITHUB_PAT, I get a Failed to add secret. Name is invalid error. I can create other ones w/o issue. – Underworld

@im1dermike Actions secrets in GitHub Settings doesn't seem to allow adding any secret environment variables prefixed with GITHUB_. You could try using the existing GITHUB_TOKEN, e.g. GITHUB_PAT: ${{ secrets.GITHUB_TOKEN }} or try naming it something else, e.g. GITHUB_PAT: ${{ secrets.MY_GITHUB_PAT }} – Serin

Answer 2

Its now possible to use simpler configuration by using checkout@v2 and setup-node@v2

jobs:
  build:

    runs-on: ubuntu-latest
    env:
      NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}
    steps:

    # 1. provide Personal Access Token for checkout@v2
    - name: Checkout
      uses: actions/checkout@v2
      with:
          submodules: recursive
          token: ${{ secrets.PUBLISH_PAT }}

    # 2. setup .npmrc it uses NODE_AUTH_TOKEN
    - name: Setup .npmrc file for publish
      uses: actions/setup-node@v2
      with:
        node-version: '12.x'
        registry-url: 'https://registry.npmjs.org'

    # 3. configure git user used to push tag
    - name: Configure Git User
      run: |
        git config --global user.email "[email protected]"
        git config --global user.name "ci@$GITHUB_ACTOR"

    - name: Install dependencies
      run: yarn install

    - name: Publish
      run: |
        lerna publish --yes

Setup repository secret with below:

NPM_TOKEN is NPM token with publish permission, more info

PUBLISH_PAT is github personal access token with repo permission, more info

Gallaway

this actually solved it for me. Just a small notice: You can use the already available secrets.GITHUB_TOKEN instead of secrets.PUBLISH_PAT – Isolde

Answer 3

Building on @JeroenKnoops comment, using checkout@v2, a simpler approach can be had:

name: lerna publish

on:
  push:
    branches:
      - master

jobs:
  publish:
    runs-on: ubuntu-latest
    env:
      NPM_TOKEN: ${{ secrets.GITHUB_TOKEN }}
      GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
    steps:
      - uses: actions/checkout@v2
        with:
          fetch-depth: 0
      - name: Read node version
        run: echo ::set-output name=nodever::$(cat .nvmrc)
        id: nvm
      - name: Setup node
        uses: actions/setup-node@v1
        with:
          node-version: '${{ steps.nvm.outputs.nodever }}'
          registry-url: https://npm.pkg.github.com/
      - name: Configure Git User
        run: |
          git config --global user.email "[email protected]"
          git config --global user.name "@$GITHUB_ACTOR"
      - run: npx lerna publish --conventional-commits --yes

Note that in this example I have a .npmrc configured which references the NPM_TOKEN environment variable to authenticate:

@myco:registry=https://npm.pkg.github.com/
//npm.pkg.github.com/:always-auth=true
//npm.pkg.github.com/:_authToken=${NPM_TOKEN}

Chaille

Could you please also show the npmrc? Both answers are kind of incomplete. 🙂 – Winch

Answer 4

Below is my smallest working setup for publishing with Lerna from GitHub Actions.

This solution is consolidated experience from the answers in this thread and some comments to related issues on GitHub.

Key Points

• Using actions/checkout@v2 to pull the repo. Set fetch-depth: "0" so that it pulls all history and tags for Lerna to detect what packages have changed.
• Using actions/setup-node@v2 to set up npm. Credit to ktutnik's answer
• Running npm whoami to throw and exit if npm is misconfigured and can not publish. This will prevent Lerna from creating and pushing tags prematurely. Credit to Benny Powers's answer
• Important! If you are using npm's Automation Token, use --no-verify-access flag when publishing with Lerna. Otherwise, lerna will fail with 403 because the token will not have enough priveleges to check access via endpoints that Lerna use. Credit to dyladan's comment on GitHub

CI

name: Lerna CI

on:
  push:
    branches:
      - master


jobs:
  build:

    runs-on: ubuntu-latest
    env:
      NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }} # set this token manually
    steps:

    - name: Checkout
      uses: actions/checkout@v2
      with:
        fetch-depth: "0" # pulls all history and tags for Lerna to detect what packages changed.
        token: ${{ secrets.GITHUB_TOKEN }} # this token is available by default

    # setup .npmrc using NODE_AUTH_TOKEN
    - name: Setup .npmrc file for publish
      uses: actions/setup-node@v2
      with:
        node-version: '12.x'
        registry-url: 'https://registry.npmjs.org'

    - name: Configure Git User
      run: |
        git config --global user.email "[email protected]"
        git config --global user.name "lerna-ci@$GITHUB_ACTOR"

    - name: Check if able to publish changes
      run: npm whoami # will throw and exit if npm is not ready to publish

    - name: Install dependencies
      run: yarn install

    - name: Publish
      run: lerna publish --no-verify-access # the flag is needed if NPM_TOKEN is an Automation Token

Exportation

thanks for this answer, it helps me out with history issue for github release, fetch-depth: "0" was the key – Larkins