Checkout Submodules in another Project in Azure DevOps Repository

Question

I'm trying to checkout a submodule from another project in azure devops.

steps:

- checkout: self
  submodules: true
  persistCredentials: true
  clean: true

Checking out another repository in the same project works.

GOAL

Add as submodule REPOSITORY A in REPOSITORY B.

However i get the error:

Cloning into 'E:/build/Agent5/_work/16/s/.azuredevops'...
fatal: could not read Username for 'https://dev.azure.com': terminal prompts disabled
fatal: clone of 'https://dev.azure.com/ORGANIZATION/PROJECTA/_git/REPOSITORYA' into submodule path 'E:/build/Agent5/_work/16/s/.azuredevops' failed

How can i achieve this using YAML in Azure DevOps Repositories?

Furred

Answer 1

When adding a submodule i found out that the url in the .gitmodules were not correct.

path = .azuredevops
url = https://dev.azure.com/ORGANIZATION/PROJECTA/_git/REPOSITORYA

Should be:

path = .azuredevops
url = https://[email protected]/ORGANIZATION/PROJECTA/_git/REPOSITORYA

Furred

Hey @Bruno where to find .gitmodules and update path I am not using submodules: true – Microsporophyll

Hey @NaveenKumar, the submodules only appear after you add a submodule. If you have already added it it should be in a hidden file in the root of your repository, try also to select in Windows folder the option to show hidden files if it does not appear. – Furred

Interestingly, the first one does work locally, but only the second one also on the Azure servers. :-/ – Fanfaron

@Furred I realize it's been years since you posted this answer, but I'd love to know more about how you discovered this. – Pigg

Answer 2

There was on option "Limit job authorization" that was checked on by default, in the Project Settings (bottom of the the left menu).

Turning it off solved the issue.

And FYI I just had this to be added in the yaml, under steps::

- checkout: self
  submodules: true

Individualist

Answer 3

I had turned off the "Limit job authorization scope" settings on an Organizational level, but was still being blocked. It turns out you need to turn them off for each individual Project as well. It does NOT get inherited.

Decelerate

I appreciate your username :P – Shay

Answer 4

i think this means pipeline doesnt have access to repo B in project B. but I dont think docs say it is possible across projects (although I do notice it says same Azure Devops Organization in couple of places, instead of same Project), but it might be, you can follow thit link:

https://learn.microsoft.com/en-us/azure/devops/pipelines/repos/pipeline-options-for-git?view=azure-devops#authenticated-submodules

In case that doesnt work you'd need to use a scripted approach get a PAT token and use that for the checkout:

https://learn.microsoft.com/en-us/azure/devops/pipelines/repos/pipeline-options-for-git?view=azure-devops#alternative-to-using-the-checkout-submodules-option

Kwiatkowski

Answer 5

Assuming that all your repos are in the same organization, and there is no policy or security blocking checking out repos between project, then you could achieve on two ways

1. Using git submodule update

In this method we are using System.AccessToken to authenticate against https repositories specified in .gitmodules file. BTW. all submodules MUST be on HTTPS!

pool :
    vmImage : "ubuntu-latest"

stages :
  - stage : Build
    jobs :
      - job : Build
        steps :
          - script: |
              git -c http.extraheader="AUTHORIZATION: bearer $(System.AccessToken)" \
                  submodule update --init --recursive
            displayName: Fetch Submodules

2. Using repositories task

Below pipeline will checkout repositories and mount them at the path locations. Path /s is important because, it will checkout repo into code working directory, and if Project A file system contains libs/repo-one/ and libs/RepoTwo/ paths, then source code of Project B will be mounted there.

pool :
  vmImage : "ubuntu-latest"

resources:
  repositories:
  - repository: repo-one
    name: Project B/repo-one
    type: git
 
  - repository: RepoTwo
    name: Project B/Repo Two
    type: git
  
stages :
  - stage : Build
    jobs :
      - job : Build
        steps :
          # Project A
          - checkout: self
            submodules: false
            path: "s/"
          # Project B
          - checkout: repo-one
            path: "s/libs/repo-one/"
          - checkout: RepoTwo
            path: "s/libs/RepoTwo/"
          
          - script: |
              ls -la .
              ls -la libs/repo-one
              ls -la libs/RepoTwo
            displayName: Debug File System

Gona