Self updating bash script from github

Question

I am trying to make my script to check if there is an update from my repo in github and then get the updates and replace old code with new one and run the new code “not the old one”. I came up with this but it updates after it finishes

self_update() {
    cd $(dirname $0)
    git fetch > a.txt 1> /dev/null 2> /dev/null
    git reset --hard >> a.txt 1> /dev/null 2> /dev/null
    git pull >> a.txt 1> /dev/null 2> /dev/null
    rm a.txt
    chmod +x "$(basename $0)"
    cd -
}
self_update
echo “some code”

Edit: I found the below code here stackoverflow and it updates my script. However, it goes into a loop and never runs the new or old code, not sure why.

#!/bin/bash

SCRIPT=$(readlink -f "$0")
SCRIPTPATH=$(dirname "$SCRIPT")
SCRIPTNAME="$0"
ARGS="( $@ )"
BRANCH="master"

self_update() {
    cd $SCRIPTPATH
    git fetch

    [ -n $(git diff --name-only origin/$BRANCH | grep $SCRIPTNAME) ] && {
        echo "Found a new version of me, updating myself..."
        git pull --force
        git checkout $BRANCH
        git pull --force
        echo "Running the new version..."
        exec "$SCRIPTNAME" "${ARGS[@]}"

        # Now exit this old instance
        exit 1
    }
    echo "Already the latest version."
}
self_update
echo “some code”

Repeated Output:

 Found a new version of me, updating myself...
 HEAD is now at 5dd5111 Update tool
 Already up to date
 Already on ‘master’
 Your branch is up to date with origin/master

It does not stop printing the output till i CTRL-C Output: Executed with: bash -x /opt/script/firstScript -h

++ readlink -f /opt/script/firstScript
+ SCRIPT=/opt/script/firstScript  
++ dirname /opt/script/firstScript
+ SCRIPTPATH=/opt/script                                
+ SCRIPTNAME=/opt/script/firstScript
+ ARGS='( -h )'
+ BRANCH=master
+ self_update      
+ cd /opt/script
+ git fetch                                                
++ git diff --name-only origin/master
++ grep /opt/script/firstScript
+ '[' -n ']'                                               
+ echo 'Found a new version of me, updating myself...'
Found a new version of me, updating myself...
+ git pull --force 
Already up to date.  
+ git checkout master
Already on 'master'
Your branch is up to date with 'origin/master'.
+ git pull --force
Already up to date.
+ echo 'Running the new version...'
Running the new version...
+ exec bash -x /opt/script/firstScript '( -h )'
++ readlink -f /opt/script/firstScript
+ SCRIPT=/opt/script/firstScript
++ dirname /opt/script/firstScript
+ SCRIPTPATH=/opt/script
+ SCRIPTNAME=/opt/script/firstScript
+ ARGS='( ( -h ) )'
+ BRANCH=master
+ self_update
+ cd /opt/script
+ git fetch
++ git diff --name-only origin/master
++ grep /opt/script/firstScript
+ '[' -n ']'
+ echo 'Found a new version of me, updating myself...'
Found a new version of me, updating myself...
+ git pull --force
Already up to date.
+ git checkout master
Already on 'master'
Your branch is up to date with 'origin/master'.
+ git pull --force
Already up to date.
+ echo 'Running the new version...'
Running the new version...
+ exec bash -x /opt/script/firstScript '( ( -h ) )'
++ readlink -f /opt/script/firstScript
+ SCRIPT=/opt/script/firstScript
++ dirname /opt/script/firstScript
+ SCRIPTPATH=/opt/script
+ SCRIPTNAME=/opt/script/firstScript
+ ARGS='( ( ( -h ) ) )'
+ BRANCH=master
+ self_update
+ cd /opt/script
+ git fetch
++ git diff --name-only origin/master
++ grep /opt/script/firstScript
+ '[' -n ']'
+ echo 'Found a new version of me, updating myself...'
Found a new version of me, updating myself...
+ git pull --force
Already up to date.
+ git checkout master
Already on 'master'
Your branch is up to date with 'origin/master'.
+ git pull --force
^C

Output: Executed with: bash /opt/script/firstScript -h

Found a new version of me, updating myself...
Already up to date.
Your branch is up to date with 'origin/master'.
Already up to date.
Running the new version...
Found a new version of me, updating myself...
Already up to date.
Your branch is up to date with 'origin/master'.
Already up to date.
Running the new version...
Found a new version of me, updating myself...
Already up to date.
Your branch is up to date with 'origin/master'.
Already up to date.
Running the new version...
Found a new version of me, updating myself...
Already up to date.
Your branch is up to date with 'origin/master'.
Already up to date.
Running the new version...
Found a new version of me, updating myself...
Already up to date.
Your branch is up to date with 'origin/master'.
Already up to date.
Running the new version...
Found a new version of me, updating myself...
Already up to date.
Your branch is up to date with 'origin/master'.
Already up to date.
Running the new version...
Found a new version of me, updating myself...
Already up to date.
Your branch is up to date with 'origin/master'.
Already up to date.
Running the new version...
Found a new version of me, updating myself...
^C

Finery

Do you get any output if you run manually, after doing the pull, the command git diff --name-only origin/$BRANCH | grep $SCRIPTNAME? – Province

I added the output, please check it out – Finery

You have only added the output of your script, not what I asked for. – Province

if i do git fetch; git reset --hard; git pull; manually. then, git diff --name-only origin/$BRANCH | grep $SCRIPTNAME will not print my script name but If i do not do it manually it it will output my script name and that make it go into a loop – Finery

So the problem is the git command. Maybe someone with more git experience than I have, can now see what's going on. I would, for testing, put a git diff --name-only origin/$BRANCH just before the if in your script, and add the output to your question. – Province

You do a git checkout $BRANCH before invoking yourself again. Hence on the next iteration, you are on $BRANCH, while on the first iteration, you are not on $BRANCH. I don't know whether this matters, though..... – Province

Does your script have local changes? The git pull would not destroy them, and afterwards git diff will still see something modified in the script (I think). Also, even if you get it working, I don't understand the reason of the logic of your approach: If one file (your script) shows diffs, you want to merge changes for all files (git pull), but if some other files shows diffs, you don't want to do this? – Province

It does not have local changes. I only made changes in github. I am going with this logic because I want to update other files thats why i am using master "as I was told". to get changes for all folders and tools – Finery

So you have to include the result of the get diff command and hope that someone with decent git-knowledge sees your question. And, I suggest to add a git tag, since the essence of your question is about using git, not bash. – Province

run your script as bash -x scriptname (and post output here) this can give you (and us) an idea of what is happening – Landers

please checkout the output @Yury Nevinitsin – Finery

Why do you redirect output with >a.txt 1>/dev/null and delete the file afterwards? There will not be any output left for >/dev/null. Just remove the >a.txt and >>a.txt and rm a.txt. – Demetricedemetris

Answer 1

After reading bash -x output I can give you a rewrite of the script

#!/bin/bash
                                               # Here I remark changes

SCRIPT="$(readlink -f "$0")"
SCRIPTFILE="$(basename "$SCRIPT")"             # get name of the file (not full path)
SCRIPTPATH="$(dirname "$SCRIPT")"
SCRIPTNAME="$0"
ARGS=( "$@" )                                  # fixed to make array of args (see below)
BRANCH="master"

self_update() {
    cd "$SCRIPTPATH"
    git fetch

                                               # in the next line
                                               # 1. added double-quotes (see below)
                                               # 2. removed grep expression so
                                               # git-diff will check only script
                                               # file
    [ -n "$(git diff --name-only "origin/$BRANCH" "$SCRIPTFILE")" ] && {
        echo "Found a new version of me, updating myself..."
        git pull --force
        git checkout "$BRANCH"
        git pull --force
        echo "Running the new version..."
        cd -                                   # return to original working dir
        exec "$SCRIPTNAME" "${ARGS[@]}"

        # Now exit this old instance
        exit 1
    }
    echo "Already the latest version."
}
self_update
echo “some code”

Below 1. ARGS="( $@ )" definitely should be ARGS=( "$@" ), otherwise after update script is executed with '( -h )' argument instead of -h (in general, it is executed with all arguments concatenated in a single string, i.e. you run it as /opt/script/firstScript -a -b -c, and after update it runs as /opt/script/firstScript '( -a -b -c )'

Below 2. Double quotes are necessary around $(...), otherwise [ -n uses ] as input argument and returns true because it is not empty (while the empty output of git-diff|grep is ignored in argument list of [ -n) (That was the loop cause)

Landers

SCRIPTPATH="$(dirname "$SCRIPT")" ---Do you mean SCRIPTPATH="$(dirname "$SCRIPTFILE")" and I added git reset --hard and chmod +x "$SCRIPTNAME" but still go into a loop – Finery

you are right, updated answer with proper SCRIPTPATH. In my environment it worked. Can you run it with bash -x again and post output again? (maybe use pastebin.com for it not to flood here) – Landers

would recommend branch=$(git rev-parse --abbrev-ref HEAD) rather than hard-coding the branch name – Catherinacatherine

and UPSTREAM=$(git rev-parse --abbrev-ref --symbolic-full-name @{upstream}) and then use "$UPSTREAM" in place of "origin/$BRANCH" – Catherinacatherine

Answer 2

There could be multiple reasons that the script might fail to update itself. Putting aside the "why" for a minute, consider using an "double-invoke" guard based on environment variable. It will prevent repeated attempt to update.

self_update() {
    [ "$UPDATE_GUARD" ] && return
    export UPDATE_GUARD=YES

    cd $SCRIPTPATH
    git fetch

    [ -n $(git diff --name-only origin/$BRANCH | grep $SCRIPTNAME) ] && {
        echo "Found a new version of me, updating myself..."
        git pull --force
        git checkout $BRANCH
        git pull --force
        echo "Running the new version..."
        exec "$SCRIPTNAME" "${ARGS[@]}"

        # Now exit this old instance
        exit 1
    }
    echo "Already the latest version."
}

Also, consider changing the self_update to go back into the original cwd, if it might have impact on running the script.

Adolphadolphe

it updates but continues using old code "not the updated code" and does not pass args. Please explain more about what you changed. – Finery

Answer 3

My suspicion is that you have not set up upstream tracking for the current branch. Then git fetch does nothing at all. Try

git fetch origin master

instead (assuming that's the upstream and branch you want).

You also don't seem to understand the significance of the exec in the code you found. This will replace the currently executing script with the updated version and start running it from the start. There is no way that

update_code
echo "some stuff"

will echo "some stuff" right after updating itself. Instead, it will execute itself again, hopefully then this time with an updated version of the code.

However,

[ -n $(git diff --name-only origin/$BRANCH | grep $SCRIPTNAME) ]

is a really roundabout and potentially brittle construct. You are asking whether grep returned any (non-empty) output ... but obviously, grep itself is a tool for checking whether there are any matches. Additionally, using SCRIPTNAME here is brittle -- if the script was invoked with a path like /home/you/work/script/update_self_test that's what SCRIPTNAME contains, but git diff will only output a relative path (script/update_self_test in this case if /home/you/work is the Git working directory) and so the grep will fail. (In your bash -x transcript, you see grep /opt/script/firstScript -- that's exactly this bug.)

Since you are in the directory of the file already, I would advocate

git diff --name-only origin/master "$(basename "$SCRIPTNAME")"

which will print nothing if the file has not changed, and the file name of this single file if it has. Unfortunately, this does not set its exit code to indicate success or failure (I guess it's hard to define that here, though the traditional convention is for the regular diff command to report a non-zero exit code when there are differences) but we can use

git diff --name-only origin/master "$(basename "$SCRIPTNAME")" | grep -q . &&

for the whole conditional. (Notice also When to wrap quotes around a shell variable?)

Finally, your own attempt has another problem as well. Besides failing to exec, you have ambiguous redirects. You try to send stuff to the file a.txt but you are also trying to send the same stuff to /dev/null. Which is it? Make up your mind.

For what it's worth,

echo testing >a.txt 1>/dev/null

sends testing to /dev/null; it first redirects standard output to a.txt but then updates the redirect, so you simply end up creating an empty file in a.txt if it didn't already exist.

Eventually, you should probably also switch to lower case for your private variables; but I see that you simply copied the wrong convention from the other answer.

Congress

Answer 4

Based on all the answers here, and some of my own requirements:

• timeout git(!)
• silent, no output
• stash local changes
• Recursive loop trap
• always use origin/main
• use git diff exitcodes instead of output

I made the following version:

#!/bin/bash
                                               # Here I remark changes
#
SCRIPT="$(readlink -f "$0")"
SCRIPTFILE="$(basename "$SCRIPT")"             # get name of the file (not full path)
SCRIPTPATH="$(dirname "$SCRIPT")"
SCRIPTNAME="$0"
ARGS=( "$@" )                                  # fixed to make array of args (see below)

self_update() {
    [ "$UPDATE_GUARD" ] && return
    export UPDATE_GUARD=YES
    
    cd "$SCRIPTPATH"
    timeout 1s git fetch --quiet

                                               # in the next line
                                               # 1. added double-quotes (see below)
                                               # 2. removed grep expression so
                                               # git-diff will check only script
                                               # file

    timeout 1s git diff --quiet --exit-code "origin/main" "$SCRIPTFILE"
    [ $? -eq 1 ] && {
        #echo "Found a new version of me, updating myself..."
        if [ -n "$(git status --porcelain)" ];  # opposite is -z
        then 
            git stash push -m 'local changes stashed before self update' --quiet
        fi
        git pull --force --quiet
        git checkout main --quiet
        git pull --force --quiet
        #echo "Running the new version..."
        cd - > /dev/null                        # return to original working dir
        exec "$SCRIPTNAME" "${ARGS[@]}"

        # Now exit this old instance
        exit 1
    }
    #echo "Already the latest version."
}
self_update
echo "some code2"

Perceptual

Answer 5

Your script launches the 'self_update' bash function, whitch itself calls exec "$SCRIPTNAME" "${ARGS[@]}". But if I read well, $SCRIPTNAME is your script.

Your script will keep calling itself recursively. That's why you are looping.

Have you considered running your script with something like cron instead of making it call itself ?

Edit: Also the git command in the test, git diff --name-only origin/$BRANCH would respond with line containing the script file if you had local changes in it, and loop forever.

Giralda

I thought too that this is the reason, but the OP tried to avoid this case by guarding the recursive call with the if... That's why asked for the output of the grep in my comment. – Province

I'll edit my answer with another remark on the git command: it answers true when there are local changes in the script and will loop forever. – Giralda

So do you mean I should add cron job in my script that schedule one Time Task to exit my script and replcae old code with new one and pass user args and re-run my script with the now code? – Finery

@Finery : Don't get the question. adding a cron job to a script does not make sense. You can define a new cron job which calls a script of your liking, but I don't see what a cron job has to do with your use case. – Province

@Finery you don't add a cron job to ths script, you run your script using cron. this job is then ran periodically and you avoid the recursive call, which is dangerous (so dangerous as if you make a mistake in the [ ] test your program loops). – Giralda