How to lint all the files recursive while printing out only files that have an error?

Question

I want to lint all the files in the current (recursive) directory while printing out only files that have an error, and assign a variable to 1 to be used after the linting is finished.

#!/bin/bash

lint_failed=0
find . -path ./vendor -prune -o -name '*.php' | parallel -j 4 sh -c 'php -l {} || echo -e "[FAIL] {}" && lint_failed=1';

if [ "$lint_failed" -eq "1" ]; then
    exit 1
fi

Example:

[FAIL] ./app/Model/Example.php

The above code doesn't find any errors, but if I run php -l ./app/Model/Example.php an error is returned.

Void

Is github.com/JakubOnderka/PHP-Parallel-Lint missing anything from your needs? It doesn't appear to do a simple list of fails, but you may be able to parse it out of the Json output with a tool like jq – Pneumato

"an error is returned" - what does that mean? If running php -l on that specific file yields an error, I would assume the rest of your code is working fine? Also, where in the given code are you calling phplint? – Holst

Answer 1

The parallel command already does what you want: it exits 0 if all jobs exit 0, and it exits non-zero if any one job exits non-zero. parallel's exit options are configurable, see the EXIT STATUS section of man parallel for details.

In your script, the use of || echo obscures the exit status of the jobs, but you can expose this again doing something like this (tested bash 4.4.7 on ubuntu):

#!/bin/bash

php_lint_file()
{
    local php_file="$1"
    php -l "$php_file" &> /dev/null
    if [ "$?" -ne 0 ]
    then
        echo -e "[FAIL] $php_file"
        return 1
    fi
}

export -f php_lint_file

find . -path ./vendor -prune -o -name '*.php' | parallel -j 4 php_lint_file {}

if [ "$?" -ne 0 ]
then
    exit 1
fi

Bandore

I had to set this aside for a while and am just coming back to it. It's close but I found that $? increments on the amount of errors linting finds (If 3 files fail then $? is 3. I can get around this with if [ "$?" -ne 0 ]; then exit 1 fi at the end of the script. – Void

@MichaelDelle excellent, I've updated my answer to reflect this. – Bandore

Great! I'll put this into my CI pipeline later and test it more. Thank you! :) – Void

Found another problem: It's creating so many processes. pastebin.com/raw/gnj2mdhs – Void

I would expect 2 processes per parallel job (one for the script and one for php -l) so 8 in total for the example script, but no more than that. Do you have multiple CI build steps happening at the same time? – Bandore

No, this was on dev, I just didn't notice it before. It's starting 13 processes :( 4 of them are for php -l – Void

This doesn't work on MacOS due to the parallel command being missing. I don't care about running this in parallel, is there a more robust version of this? – Wagshul

Answer 2

You can use PHP Parallel Lint tool which checks the syntax of PHP files faster and with a fancier output by running parallel jobs while printing out only files with the errors.

Example usage:

./bin/parallel-lint --exclude app --exclude vendor .

Or using Ant's build.xml:

<condition property="parallel-lint" value="${basedir}/bin/parallel-lint.bat" else="${basedir}/bin/parallel-lint">
    <os family="windows"/>
</condition>

<target name="parallel-lint" description="Run PHP parallel lint">
    <exec executable="${parallel-lint}" failonerror="true">
        <arg line="--exclude" />
        <arg path="${basedir}/app/" />
        <arg line="--exclude" />
        <arg path="${basedir}/vendor/" />
        <arg path="${basedir}" />
    </exec>
</target>

Versicle

Answer 3

Since version 8.3 you can use glob patterns for file paths

php -l src/*.php src/**/*.php

https://php.watch/versions/8.3/cli-lint-multiple-files#glob

But for me work with enabled globstar in bash:

shopt -s globstar

https://unix.stackexchange.com/a/379192/334480

Deuteranopia