So we're using Gerrit Trigger (2.23.0) on our Jenkins CI build manager & using docker containers for the actual builds.
The issue that has recently popped up in some of our branches the Gerrit Repo polling is failing and causing it to "detect changes" every time, so it's constantly rebuilding despite no changes.
Checking the Gerrit Repo Polling Log for any of the affected jobs gives one of the following outputs:
Started on Feb 1, 2017 3:12:25 PM
Polling SCM changes on aosp-host
[workspace] $ repo init -u http://xxx.xxx.xxx.xxx/git/project/platform/manifest.git -b branch -m branch.xml
Get https://gerrit.googlesource.com/git-repo/clone.bundle
Get https://gerrit.googlesource.com/git-repo
fatal: Not a git repository: '/home/jenkins/workspace/.repo/manifests.git'
fatal: Not a git repository: '/home/jenkins/workspace/.repo/manifests.git'
fatal: cannot obtain manifest http://xxx.xxx.xxx.xxx/git/project/platform/manifest.git
Done. Took 1 min 19 sec
Changes found
or, if the build was already building (gerrit waits for the build to finish before doing the scm poll)
Started on Feb 2, 2017 3:24:15 AM
Polling SCM changes on aosp-host
[workspace] $ repo init -u http://xxx.xxx.xxx.xxx/git/project/platform/manifest.git -b branch -m branch.xml
fatal: cannot make /home/jenkins/workspace/.repo/repo directory: File exists
Done. Took 2 hr 4 min
Changes found
The builds, which are triggered by this failure, use the same commands and work fine:
[workspace] $ repo init -u http://xxx.xxx.xxx.xxx/git/project/platform/manifest.git -b branch -m branch.xml
Navigating to the manifest directory, we see the symptom:
jenkins@f052b3453d95:~/workspace/.repo$ ll
total 32
drwxr-xr-x 1 jenkins jenkins 180 Dec 20 11:08 ./
drwxrwxr-x 1 jenkins jenkins 778 Dec 20 11:07 ../
-rw-r--r-- 1 jenkins jenkins 20087 Dec 20 10:14 .repo_fetchtimes.json
lrwxrwxrwx 1 jenkins jenkins 20 Dec 20 10:13 manifest.xml -> manifests/branch.xml
drwxr-xr-x 1 jenkins jenkins 8 Dec 16 17:33 manifests/
drwxr-xr-x 1 jenkins jenkins 50 Dec 16 17:33 manifests.git/
drwxr-xr-x 1 jenkins jenkins 28 Dec 16 17:43 project-objects/
-rw-r--r-- 1 jenkins jenkins 7756 Dec 20 10:14 project.list
drwxr-xr-x 1 jenkins jenkins 410 Dec 16 17:46 projects/
with
lrwxrwxrwx 1 jenkins jenkins 20 Dec 20 10:13 manifest.xml -> manifests/branch.xml
highlighted in red, because the associated branch.xml is not found... so the Gerrit Log from above is accurate, it's failing to init properly. This is confirmed via a repo status in the main directory:
jenkins@f052b3453d95:~/workspace$ repo status
Traceback (most recent call last):
File "/home/jenkins/workspace/.repo/repo/main.py", line 531, in <module>
_Main(sys.argv[1:])
File "/home/jenkins/workspace/.repo/repo/main.py", line 507, in _Main
result = repo._Run(argv) or 0
File "/home/jenkins/workspace/.repo/repo/main.py", line 180, in _Run
result = cmd.Execute(copts, cargs)
File "/home/jenkins/workspace/.repo/repo/subcmds/status.py", line 130, in Execute
all_projects = self.GetProjects(args)
File "/home/jenkins/workspace/.repo/repo/command.py", line 140, in GetProjects
all_projects_list = manifest.projects
File "/home/jenkins/workspace/.repo/repo/manifest_xml.py", line 350, in projects
self._Load()
File "/home/jenkins/workspace/.repo/repo/manifest_xml.py", line 407, in _Load
self.manifestProject.worktree))
File "/home/jenkins/workspace/.repo/repo/manifest_xml.py", line 443, in _ParseManifestXml
root = xml.dom.minidom.parse(path)
File "/usr/lib/python2.7/xml/dom/minidom.py", line 1918, in parse
return expatbuilder.parse(file)
File "/usr/lib/python2.7/xml/dom/expatbuilder.py", line 922, in parse
fp = open(file, 'rb')
IOError: [Errno 2] No such file or directory: '/home/jenkins/workspace/.repo/manifest.xml'
The issue is, running repo init -u <url> -m branch.xml via the command line works fine, and produces a valid repo.
Any insight one can offer for this issue?