Using an array of values to repeat a step in GitHub Actions workflow

Question

I am trying to create a GitHub Actions workflow which would collect specific paths changed in last commit and run a step for each of collected paths, if any.

Currently, in my workflow I'm creating an array of paths, but I'm not sure how to proceed with my array:

name: Test

on:
  push

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v1

      # This step will create "an array" of strings, e.g. "path1 path2 path3"
      - name: array
        id: arr
        run: |
          arr=()
          for i in "$(git diff-tree --no-commit-id --name-only -r ${{ github.sha }})"
          do
            if [[ $i == *"path1"* ]]; then
              arr+=("path1")
            fi
            if [[ $i == *"path2"* ]]; then
              arr+=("path2")
            fi
          done
          echo ::set-output name=arr::${arr[@]}

      # How to run this step by iterating the `${{ steps.arr.outputs.arr }}`?
      - name: reviewdog-lint
        uses: reviewdog/action-eslint@v1
        with:
          github_token: ${{ secrets.GITHUB_TOKEN }}
          reporter: github-pr-review
          eslint_flags: 'my_project/some_folder/${{ SINGLE_PATH }}/'  # `SINGLE_PATH` would be a path from the array

Is something like this even possible in the first place? If not, what would be recommended way to loop through some values and use them as arguments in other workflow steps?

Jessen

Maybe this is a good alternative github.com/marketplace/actions/file-changes-action for getting the paths that have changed in the last commit – Kangaroo

Answer 1

Difficult to say without running it, but I would say you need to use the output in the second step by assigning it to a variable, something like:

env:
          OUTPUT: ${{ steps.id.outputs.arr }}

Then you use $OUTPUT as an environment variable inside the action.

The problem with that particular action is that it takes one commit at a time. But you can check out the code, it's a shell script. You can fork it from line 15 and make it split input and run a loop over it, applying eslint to every one of them.

Mosera

I can use ::set-output or ::set-env in my first step, there is no issue with that, but no matter what I set (output or env variable) I'm not sure how to repeat the second step for each value in my array. – Jessen

@Jessen first, GitHub actions offer no such mechanism that I know of, although it would in principle be possible to generate events recursively, you would need to do that editing code; the thing is that as long as you're editing code, you could as well fork the used action. – Mosera

Thanks for the explanation! Can you please just expand your answer a bit more, you said that "there is no such mechanism in GitHub Actions", but apart from modifying the action itself (as you suggested and which I will most likely do) and to avoid doing the same for any other action, is there some alternative method in workflow .yml file itself? – Jessen

@Jessen Thanks for accepting the answer. Unfortunately, that does not seem to be the case. There's very limited control flow in the action syntax. You can use 'if', but not create steps dynamically, for instance. Another option is to run the shell script directly, since github actions can do that too. – Mosera

I have a similar requirement. However, in my case, I need to loop through some yaml values (AWS account IDs) and access secrets based on those IDs. But neither is there a way to access secrets based on dynamic variables nor there seems to be a way to loop the steps. GitHub Actions is such a big disappointment. – Jape

Another problem with this solution is it assigns arrays to env variables, although arrays are bash-specific and env variables are (key, value) couples where the values are string, thus can't be arrays. This implies the array shall be transformed to string first (using a separator character) and transformed back afterward, which could be prone to errors depending on the array content. – Lore

Answer 2

There is some support for this in GitHub Actions. There is a very good tutorial here that explains how to do it in detail, but essentially what you'll do is split the steps into two jobs. The first job will output a JSON object that will serve as the input to the matrix of the second job.

Here's a simple example:

name: Test Github Actions Dynamic Matrix
on:
  workflow_dispatch:
  
jobs:
  setup:
    runs-on: ubuntu-latest
    outputs:
      matrix: ${{ steps.matrix.outputs.value }}
    steps:
      - id: matrix
        run: |
          echo "value=[\"a\", \"b\", \"c\"]" >> $GITHUB_OUTPUT
      - run: |
          echo "${{ steps.matrix.outputs.value }}"
  build:
    needs: [ setup ]
    runs-on: ubuntu-latest
    strategy:
      matrix:
        value: ${{fromJSON(needs.setup.outputs.matrix)}}
    steps:
      - run: |
          echo "${{ matrix.value }}"

You can see it action here

Refraction

Not really an answer because that is serialization of different jobs, not steps. Each of this job will run on a different runner and will not be able to use results from previous steps. – Tjader

This isn't working (anymore): Error when evaluating 'strategy' for job 'build'. .github/workflows/blank.yml (Line: 20, Col: 16): Error parsing fromJson,.github/workflows/blank.yml (Line: 20, Col: 16): Unexpected character encountered while parsing value: \. Path '', line 1, position 1.,.github/workflows/blank.yml (Line: 20, Col: 16): Unexpected type of value '', expected type: Sequence. – Polysaccharide

I concur with @Polysaccharide this no longer works. Tried to use this when using an environment variable as the array values. – Lateen

I've updated the example to account for the removal of the "::set-output::" syntax – Refraction