VSTS Build : Git submodule automatization

Question

We switched from TFS to GIT. We are trying to update the submodule everytime we launch a new build.

We followed this guide : https://www.visualstudio.com/en-us/docs/build/scripts/git-commands#enable

We have an error at line 49.

We think that actually we need to authenticate. But we arent sure. We used : git pull and it works but when we do this : git submodule foreach git pull origin master. We have the message "Entering" and nothing happens

Did somebody already have this problem ? How did you solve it ?

Chitter

Where does the submodule git repo hosted, the same account on VSTS or github etc? And what if you use powershell task with two lines script: cd <submodule folder> and git push origin master? – Average

They are hosted on the same VSTS account. Actually we want to use this cmd : git submodule foreach git pull origin master. And you dont know really why its doesnt work. – Chitter

Answer 1

It is an authentication issue. You need to get the OAuth token into each of the submodule repos.

Make sure you have the build definition setting enabled to Allow scripts access to the OAuth token. As documented, this stuffs the token into a variable called System.AccessToken. It also stuffs the token into a git config setting that you'll see at the end of your get sources step when you run it after enabling the setting. This is how git authenticates to VSTS. You'll need to build a config statement for each of the repos, and you'll need to cd into that submodule to issue it in that repo.

Here is the Powershell script I used:

$mods = (git submodule status) | % { ($_.Trim() -split " ")[1] }
$baserepo = ($env:BUILD_REPOSITORY_URI).TrimEnd($env:BUILD_REPOSITORY_NAME)
foreach($mod in $mods)
{
cd $mod
$cmd = 'git config http.' + $baserepo + $mod + '.extraheader "AUTHORIZATION: bearer ' + $env:System_AccessToken + '"'
write $cmd
iex $cmd
cd ..
}

Then run a cmd or powershell step:

git submodule update --remote

Lastly, you should clean up the token after you are done with it, so the OAuth doesn't hang out in your .git/config file on your build agent:

$mods = (git submodule status) | % { ($_.Trim() -split " ")[1] }
$baserepo = ($env:BUILD_REPOSITORY_URI).TrimEnd($env:BUILD_REPOSITORY_NAME)
foreach($mod in $mods)
{
cd $mod
$cmd = 'git config --unset-all http.'+ $baserepo + $mod + '.extraheader'
write $cmd
iex $cmd
cd ..
}

Planksheer

This is an awesome workaround. Note TrimEnd probably is the wrong call (use Replace maybe?) - TrimEnd takes a character array rather than a string and trims any of those characters. – Chanellechaney

Answer 2

It seems caused by the way VSTS build handle with git command with submodules.

I posted VSTS build hang up when execute git commands related to git submoudle, you can follow up the issue.

BTW, for the OAuth, you can add credential in .gitmodules.

Average

It looks like our problem. It's the same we have. I'll follow up your post, hopefully there is a way to resolve this. Thanks :) – Chitter

Answer 3

Just found this Q/A with googling: You can use relative paths, ex:

origin  ssh://[email protected]:22/PROJECT/_ssh/REPO (fetch)

the .gitmodules can contain:

[submodule "SUBMODULE"]
    path = SUBMODULE
    url = ../SUBMODULE

Source

Have a nice day :)

Nimitz

Answer 4

In your .gitmodules file you can try to add the organization name (case sensitive) in the url

[submodule "MySubModule"]
path = MySubModule
url = https://<Organization Name>@dev.azure.com/<Organization Name>/_git/<Project Name>

Grandfather