How to run commitlint in GitHub workflow on every commit of a push

Question

I have a Github repository, installed commitlint and husky locally and would like to setup a workflow running commitlint on every commit of a push when validating pull requests. On the main branch older commits are not following the conventional commit rules.

I created a separate branch, based on this comment

https://github.com/conventional-changelog/commitlint/issues/586#issuecomment-657226800

I started with this workflow

name: Run commitlint on pull request

on: pull_request

jobs:
  run-commitlint-on-pull-request:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v2
        with:
          fetch-depth: 0

      - name: Setup Node
        uses: actions/setup-node@v2
        with:
          node-version: 14.x

      - name: Install dependencies
        run: npm install

      - name: Validate all commits from PR
        run: npx commitlint --from HEAD~${{ github.event.pull_request.commits }} --to HEAD --verbose

I made two more commits following the conventional commit rules and started a pull request

• I expected the workflow wouldn't run because I doesn't exist on the main branch yet.
  • Actually it runs
• I exptected the workflow to check PR commits only
  • The workflow fails because it starts validating EVERY commit in the main branch. And since I know older commits don't follow the rules, this will never pass.

The first solution coming to my mind would be to rebase everything and rename each commit to follow the rules but this would require a huge effort.

I'm not sure if I have to improve this line here

npx commitlint --from HEAD~${{ github.event.pull_request.commits }} --to HEAD --verbose

to check commits from the PR only (unfortunately I don't know what needs to get fixed there).

Do you have any ideas or is rebasing and renaming the only solution?

Johnnie

Try npx commitlint --from $commit --to HEAD --verbose || exit 1 – Severen

sorry, unfortunately the || exit 1 didn't help. the workflow still passes – Johnnie

Does npx commitlint exit with error codes at all? – Severen

sorry, I wasn't able to find out. But the image shows I have to fix the syntax I think? – Johnnie

IMO if you want to test commits one by one --from $commit --to HEAD is wrong, it should be one commit, something like --from $commit~ --to $commit. Or instead of the loop test all commits at once: --from ${{ github.base_ref }} --to ${{ github.head_ref }} without a loop. – Severen

I tested both suggested solutions. The first one didn't work, the error remains, the workflow passes. I will update my question to show the result of your second solution – Johnnie

Move whatever function you want to run into a shell script that exits with non-zero on failure, test it locally, then run the same script in the workflow. This approach will shorten your debugging cycle by an order of magnitude. – Betthezul

yeah the problem is that I can't test this npx commitlint --from ${{ github.base_ref }} --to ${{ github.head_ref }} --verbose outside from Github – Johnnie

git rev-list documentation suggests to use the three dots ... notation to list merges. This seems more in line with the purpose or alternatively git rev-list main test --not $(git merge-base --all main test) – Selda

@ConstantinKonstantinidis I moved from two dots to three dots as suggested. Now the error from the first image was thrown and the PR passes (shoudln't pass). I also tried your second approach and get the same result. Is this the approach you wanted to try? for commit in $(git rev-list ${{ github.base_ref }} ${{ github.head_ref }} --not $(git merge-base --all ${{ github.base_ref }} ${{ github.head_ref }} )); do – Johnnie

Answer 1

Solution

The straight-forward solution is to use the --to and --from arguments of commitlint with the SHA-1 values instead of the branch names or relative references. On the one hand, this reliably solves the problem of unknown revisions or paths in the working tree. On the other hand, only commits in scope of the PR will be checked. As a sidenote: GitHub uses the same references (SHA-1) for the ad-hoc merge that is being checked-out.

We need the base-SHA as well as the head-SHA. In a GitHub action those values are available in the pull-request object of the event in the github-context.

Therefore, you can use the following line which is tested and works as expected:

npx commitlint --from ${{ github.event.pull_request.base.sha }} --to ${{ github.event.pull_request.head.sha }} --verbose

Demo

Here is a POC repository on GitHub with 3 test cases (pull-requests).

Complete workflow

name: Run Commitlint on PR

on:
  pull_request:

jobs:

  run-commitlint-on-pr:
    runs-on: ubuntu-latest

    steps:

      - uses: actions/checkout@v2
        with:
          fetch-depth: 0

      - name: Setup Node
        uses: actions/setup-node@v2
        with:
          node-version: 14.x

      - name: Install dependencies
        run: npm install

      - name: Validate all commits from PR
        run: npx commitlint --from ${{ github.event.pull_request.base.sha }} --to ${{ github.event.pull_request.head.sha }} --verbose

Newhall

What about commits to pushed directly to master without a pull request? – Clench

That was actually poor of me to say since commits pushed to master shouldn't actually be overwritten. Sorry for asking. – Clench

@KutsanKaplan Your question has merit. You need to specify the branches in the on: section. Have a look at the official documentation. – Newhall

Answer 2

git rev-list is considered because the commit of the pull request (PR) seems invalid. No loop should be required.

This issue hints to checkout the PR branch which seems simpler than fetching the PR commits. From the question, testing on default branch does not seem required.

- uses: actions/checkout@v2
        with:
          fetch-depth: 0
          ref: ${{ github.event.pull_request.base.sha }}

The lint instruction would be:

npx commitlint --from ${{ github.event.pull_request.base.sha }} --verbose

Documentation of pull-request payload does not offer the list of commits right away.

Selda

hey, thanks for your help! :) I'm going to try out your suggestion with this workflow pastebin.com/zT8X8wLw soon – Johnnie

So I tried it out but unfortunately the lint instruction doesn't work. I uploaded the full log on imgur imgur.com/a/XFvYnK5 the short error message is: [input] is required: supply via stdin, or --env or --edit or --from and --to – Johnnie

Indeed, stdin is used when no parameter is provided. Suggested command is supplemented with --from flag. – Selda

Unfortunately that didn't help. The workflow passes although I created one valid commit message, one invalid and after that I created a valid last one. I would expect that the workflow fails because the second one is invalid. But the workflow passes. – Johnnie

You seem to imply that only the relevant commits are tested. The linting rules are not in the question. What is the expected discrepancy ? – Selda

I expect that if I create a pull request with 2 valid commits and 1 invalid commit, the process will fail and show me which commits are invalid. If I create a pull request with only valid commits, it should succeed. If I create a pull request with invalid commits only, the process should fail and show me information. – Johnnie

That is understood from above. So, linting fails on one commit but the github-action does not. Could you post the log to check details ? – Selda

does this help? pastebin.com/MTW6KXBH I will describe what I did for reproduction in the next comment – Johnnie

I setup npm, commitlint and husky as described here commitlint.js.org/#/guides-local-setup. I create the workflow file. Now I move into another branch and modify the README and make some commits. After that I can create PRs from this branch and test the functionality – Johnnie

The commitlint instruction does not seem to do much as the log is empty. git checkout occurs but the object is the head of the PR and not the base of it. There is a git log -1 instruction. It could be useful to set 4 to check that indeed all commits are available using the existing ref. – Selda

you mean like adding an additional step e.g. run: git log 4 ? – Johnnie

Answer 3

I used the following github action: https://github.com/wagoid/commitlint-github-action

From the README of the linked repository:

Create a github workflow in the .github folder, e.g. .github/workflows/commitlint.yml:

name: Lint Commit Messages
on: [pull_request]

jobs:
  commitlint:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
        with:
          fetch-depth: 0
      - uses: wagoid/commitlint-github-action@v5

Ratal

Easiest way to implement commitlint in workflows, and this avoid warnings too – Rodrique