setting up mercurial/kiln subrepos on osx

Question

I have been trying to follow the instructions in the answer to this question, using kiln.

i'd like to be able to arrange things as follows:

• /somepath/thirdparty maps to a kiln repository "thirdparty" and contains assorted code
• /somepath/common maps to a kiln repository "common" and contains shared code i have written

and

• /somepath/project1 maps to kiln repository "project1"
• /somepath/project1/thirdparty maps to branch of thirdparty above
• /somepath/project1/common maps to branch of common above

and

• /somepath/project2 maps to kiln repository "project1"
• /somepath/project2/thirdparty maps to another branch of thirdparty above
• /somepath/project2/common maps to another branch of common above

I found that when I created the .hgsub file as instructed and added/pushed it to Kiln, I could no longer view the Kiln files in the Kiln web file viewer — it displayed an obscure message about the Kiln "overheating" :-) Additionally, whilst it did automatically create the subfolders in the correct place, they were not populated with files, (possibly because the pull failed).

Anybody tried anything like this before, using Kiln?

As I intend to develop a number of apps using the common code (and potentially eventually release the library as open source), I would like to have it managed in discrete repositories. As some of the projects are for end clients however, I need to be able to give them a single repository that includes things as described above.

Petrochemical

@Martin Geisler - thanks for the edit, much easier to read now! – Petrochemical

No problem, I'm happy to tidy up a bit! :) – Adames

Answer 1

Kiln does currently not support subrepos that use nested URLs on the server. This means that you cannot have both the following URLs working:

http://server/kiln/somepath/project1
http://server/kiln/somepath/project1/thirdparty

So you should setup Kiln so that you have four repositories on the server:

http://server/kiln/somepath/project1
http://server/kiln/somepath/project2
http://server/kiln/somepath/thirdparty
http://server/kiln/somepath/common

That's easy — just four normal repositories. Then clone "project" and create the .hgsub file with:

thirdparty = http://server/kiln/somepath/thirdparty
common = http://server/kiln/somepath/common

When you push that back to Kiln, it will notice and display links for the subrepositories. However, the subrepositories wont end up being nested on the server. So there wont be any project1/thirdparty path on the server.

It's also far from clear that you would want that. When you have several projects that collaborate and use some common code base, then you want "project1" and "project2" to get each other's changes to this common code base. So it very useful that the common subrepo in both projects push and pull from http://server/kiln/somepath/common.

In Mercurial, we normally recommend that you use paths of the form common = common in the .hgsub file. This means that the server must support nested repositories. When Kiln doesn't support nested repos, you can use full paths instead.

When you initially setup the subrepositories, then remember that you need to update them manually. So with the above URLs, you would setup "project1" by running:

$ hg clone http://server/kiln/somepath/project1
$ echo "common =     http://server/kiln/somepath/common" > .hgsub
$ echo "thirdparty = http://server/kiln/somepath/thirdparty" > .hgsub
$ hg commit -m "Created subrepos"

This creates initial empty subrepositories. They are empty because you haven't told Mercurial which changeset you need in them. This is tracked in .hgsubstate where you'll find:

0000000000000000000000000000000000000000 common
0000000000000000000000000000000000000000 thirdparty

To populate the subrepositories you do

$ cd common
$ hg pull --update
$ cd ../thirdparty
$ hg pull --update
$ cd ..
$ hg commit -m "Updated subrepos"

This updates the 000... lines in .hgsubstate with the current tip changeset IDs for the two subrepos. Future clones of "project1" will notice the .hgsubstate file and make sure to update the subrepos to the revision mentioned there.

Adames

hmmm, that appears to be how i had it set up and it was failing (i didn't have nested repos on the server, but only locally). i shall have a closer look and see how my approach differs to your answer, and what might have been causing the error. thanks. – Petrochemical

as far as i can tell everything is set up the same, except it's using https:// instead of http:// (my [auth] section my .hgrc file in home folder has the appropriate login credentials for the kiln login). my actual paths have meaningfull names i don't wish to post here, but other than that it is semantically identical. – Petrochemical

Okay, strange. When you hg clone the project, Mercurial will simply hg clone the subrepos. So try cloning server/kiln/common by hand. If that doesn't work, then something is wrong with the repo in Kiln. I guess you know this, but a subrepo isn't special — it's just a normal repo on the server. Other repos can reference it in the .hgsub file, but the subrepo doesn't "know" this. – Adames

more info: when i pushed the repo back to kiln after having added the .hgsub file, i observed it had created an empty folder with the appropriate name, nested inside the project folder. i then tried to pull back the project repo, hoping it would drag in the files too. at that point i went to then kiln website and discovered the amusing "overheating" message. since reading your last comment, i tried "hg clone https://(myrepos_url_on_kiln)" to see if making a new clone of the whole thing would make any difference. result: no subfolders ie as if the push of the .hgsub did not take effect. – Petrochemical

btw: the new clone described above was in a fresh folder. – Petrochemical

Okay, your last comments hints at a possible misunderstanding: you need to pull the subrepos manually. If you don't, you end up with empty subrepos in a fresh clone. – Adames

so, does the fact that the empty folders were created by the push but not the clone indicate anything to you? ie i can't cd into a folder that is not there. – Petrochemical

ok, resolved it. what i needed to do was follow your instructions in the original folder and then push the results back to kiln, however there was a step you missed out - i needed to do "hg update" after the "hg pull" in each sub repos. thanks – Petrochemical