OS X Server - bot can't get source from repository

Question

I had a previous version of OS X Server set up and running fine, but when I installed the upgrade to 3.2.1, I found that none of my bots would work correctly. These are the issues I get on each attempted integration:

Build Service Error Can't fit data in the buffer (-1).

Build Service Warning The source control operation failed because no working copy could be found.

Build Service Warning An error occurred updating existing checkout. Falling back to a clean checkout..

I may be looking in the wrong place, but I think the errors mean that I can't connect to my Git repository. This was partially confirmed when I deleted my repository from Xcode and ran the bot again and got the same errors. (I have since re-added the repository in Xcode and performed a Check Out to test it, so am pretty certain I have all the connection details right.)

With the previous version of OS X Server, I'm sure I'd connected to the repository directly from the server, as explained in Help > Server Tutorials > Automate Xcode builds:

Step 3. Give Xcode service access to the git repository

Xcode service bots access projects and code from your source git repository.

Click Repositories, then click the Add button (+). Choose Connect to a Git Repository from the New Repository pop-up menu.

But in the latest version, when you click the Add button you just get the option to create a new repository, rather than connect to an existing one. And the instructions on the web (now) say:

Connect to Remote Repositories

If you have projects in Git or Subversion repositories on remote servers, you can store your credentials for them on your development Mac in the Accounts preferences in Xcode. Then, when you need to access the repositories, you won’t have to reenter your credentials every time.

To add a remote repository’s credentials to a development Mac

For Xcode Server to perform integrations on your projects, it must also have access to their source code repositories. Configure Xcode Server to connect to your remote repositories.

• Choose Xcode > Preferences on your development Mac.

• Click Accounts in the toolbar.

• Click the Add button (+), and choose Add Repository.

• In the text field, enter the URL for the repository (for example, svn+ssh://svn.example.com/ProjectName or https://example.com/git/repository.git), and click the Next button.

• Enter your user name and password in the Repository pane of Accounts preferences.

which is a bit vague, but seems to suggest that now OS X Server uses repositories that are set up in Xcode's preferences.

Anyway, now my OS X Server can't seem to access the repository. Am I missing something here?

Thanks

UPDATE:

I've come back to this problem after a month of pretending it didn't exist, and have made the following findings:

-The problem occurs with OS X Server 3.2.1, OS X Server 3.2.2 and OS X Server 4.0, on two different machines.

-The problem occurs when the repo is on a server running Gitblit, but not for remote Bitbucket repos, or for a Git repo hosted within a local OS X Server.

-The problem only occurs for larger repos.

Mongol

Unfortunately I cannot help, other than to say I don't think you're missing something. I can no longer point Xcode server at external SVN or GIT repositories. I really hope this is a bug, not the new implementation because it has crippled this CI system for me! – Stypsis

I've just updated to OS X Yosemite and OS X Server 4, and I'm still having the same issue! There is now also a 3.2.2 OS X Server upgrade that you can only install before you upgrade to Yosemite, so it'd be interesting to see if that fixes the issue for anyone. – Mongol

Hi João. I did - please see the accepted answer below. Thanks – Mongol

@jimmyjudas. Our repo is with bitbucket. but we saw the similar issue. – Christelchristen

Answer 1

I've solved this problem in the most convoluted way imaginable!

When OS X Server connects to the repository to download the source, it uses the Xcode library. Xcode uses libgit2 to do the downloading which had a known bug which caused the buffer error. It was fixed in v0.21.2, but the latest Xcode (6.1.1) only uses v0.21. That previous Xcode version only used v0.20, so hopefully a future Xcode release will use v0.21.2+ in which case this fix shouldn't be necessary.

Anyway, the solution for Xcode 6.1.1 was to:

1. Download and unzip the version of gitlib used by Xcode from here: https://github.com/libgit2/libgit2/releases/tag/v0.21.0
2. Make the changes to src/transports/http.c as detailed here: https://github.com/libgit2/libgit2/commit/7d729d0bfd897e8685099b160b6dbfd7f4ebd588
3. Save the http.c file

4. Build your edited version of gitlib2 using the instructions here, i.e:

   • Download CMake

   • Run the following commands in Terminal after navigating to your unzipped libgit2-0.21.0 folder:

     $ mkdir build && cd build

     $ cmake ..

     $ cmake --build .

5. In Finder on your server, go to the Applications folder, right-click on Xcode and select "Show Package Contents". Go to the Contents/Developer/usr/lib folder and rename the 'libgit2.dylib' file to 'libgit2.dylibold' or similar.

6. Take the newly created 'libgit2.0.21.0.dylib' file from the libgit2-0.21.0/build folder, rename it to 'libgit2.dylib' and copy it into the /Applications/Xcode/Contents/Developer/usr/lib folder on your server.

7. Restart OS X Server, and integrate your bot again from Xcode.

Mongol

Though it was fun patching one of Xcode's dylibs, this didn't fix it for me. It wouldn't let me get past the repository credentials screen while creating a bot, saying "This transport isn't implemented. Sorry (-1)", which I guess is some libgit2 error. Oh well. – Antonantone

Hi Travis. Sorry to hear it didn't fix it for you. It sounds like you were having a different issue to me though. – Mongol

This indeed solved my issue, but it should be noted that this is only the solution for git servers that are built on jgit. – Tishtisha

Glad to hear it @bolnad. I'm assuming only jgit servers lead to this exact error anyway. – Mongol

@TravisB, have you figured it out? I saw the same error: this transport isn't implemented. sorry (-1). – Christelchristen

Answer 2

The remote repository will be added when you create a bot. You don't have to add repository in OSX server.

Abortionist

Yes, I'd got that thanks. But the problem is that my bots don't seem to be able to get to the repositories when they integrate. – Mongol

I have a similar problem. I can add the repo in XCode (Git repository, hosted on TFS server, accessed via https), but when I Product->Create Bot it fails verification with: develop: Unexpected HTTP status code: 404 (-1) even though I can use XCode's source control to checkout the code. :| – Bertold

me 3! I get exactly the same issue! – Punnet

try to authenticate via ssh instead of https. – Abortionist