.hgignore not being parsed for a certain user

Question

I have a project under version control, say, in /project, and .hgignore located at /project/.hgignore. Its syntax seems correct, but the problem is this file is completely ignored for certain users while still parsed for the others.

Say, running

su -l dipsy -c 'cd /project; hg status'

shows correct results with proper files ignored, while

su -l laalaa -c 'cd /project; hg status'

also outputs files listed in /project/.hgignore.

What I checked already:

• ~/.hgrc files are identical for both users, so are outputs for hg showconfig.
• Both users can read /project/.hgignore and write to that.

What am I missing?

(Just in case: Debian Lenny, Mercurial 1.6.3)

// Sorry if usernames seem stupid, they are not real (:

-- added 2010-11-26 --

PS. Is there any way to launch hg and get the debug output on processing .hgignore-s? hg --debug status and hg status --debug do not print anything sensible.

-- added 2010-11026 --

Debugging hg status (results vary):

# su -l dipsy -c 'cd /project; strace hg status 2>&1 >/dev/null | grep hgignore'
open("/project/.hgignore", O_RDONLY|O_LARGEFILE) = 4
fstatat64(4, ".hgignore", {st_mode=S_IFREG|0664, st_size=214, ...}, AT_SYMLINK_NOFOLLOW) = 0
write(1, "M .hgignore\nM foo/bar/baz"..., 4096) = 4096

# su -l laalaa -c 'cd /project; strace hg status 2>&1 >/dev/null | grep hgignore'
write(1, "M .hgignore\nM foo/bar/baz"..., 4096) = 4096

Debugging hg status --ignore (results are the same):

# su -l dipsy -c 'cd /project; strace hg status --ignore 2>&1 >/dev/null | grep hgignore'
open("/project/.hgignore", O_RDONLY|O_LARGEFILE) = 3
fstatat64(3, ".hgignore", {st_mode=S_IFREG|0664, st_size=214, ...}, AT_SYMLINK_NOFOLLOW) = 0

# su -l laalaa -c 'cd /project; strace hg status --ignore 2>&1 >/dev/null | grep hgignore'
open("/project/.hgignore", O_RDONLY|O_LARGEFILE) = 3
fstatat64(3, ".hgignore", {st_mode=S_IFREG|0664, st_size=214, ...}, AT_SYMLINK_NOFOLLOW) = 0

So, /project/.hgignore is read when running hg status --ignore and skipped if running just hg status. WTF?

Cartridge

Answer 1

Answer 1 - Dirstate Repository Corruption

I just had this problem as well last night and it was driving me up the wall trying to find the cause of it. I eventually found a wiki page on dirstate repository corruption. The first step involving running hg verify didn't work. The second step, cloning the repo, did work! I then deleted the original .hg directory and copied the clone's .hg directory to the orginal location.

I'm guessing that in your answer, the steps involving commiting/pushing may have fixed the corruption in your repository.

---

Answer 2 - inotify extension bug

After I originally fixed the problem and posted my answer, the problem continued to pop up, but in a different manner: Mercurial seemed to be partially obeying the .hgignore file, but any updates I made to it did not take effect. I happened to be playing around creating a script to create several related repositories, and I noticed after a while that my machine was running out of memory. I ran a ps -e and there were all of these hg processes hanging around in memory. All of those processes were inotify servers.

Inotify is an extension, shipped with Mercurial, which subscribes to any changes in your working directory in order to improve the performance of hg status on large repositories. The inotify extension page mentions that "it definitely has to be considered experimental". It seems that some bug in the inotify servers is preventing Mercurial realizing the .hgignore file has been updated and so hg status always used a stale version of .hgignore.

To try my theory out and temporarily get hg to refresh .hgignore I executed:

killall -s 2 hg

This command tells all resident inotify servers to exit. (killall is like kill, but sends the signal to all processes with the given name. The -s 2 argument sends the INT signal which allows inotify to shutdown gracefully.)

After that, things started working very nicely, but inotify servers kept popping up after executing hg. To stop that I put the following snippet in my hgrc file:

[extensions]
hgext.inotify = !

This disables the inotify extension (the ! instructs Mercurial to disable the extension). My repositories are small enough that I don't need it for now.

Suffuse

Great, that worked. Now my recovery plan is shorter -- just hg clone /repo /newrepo; mv /repo/.hg /repo/.hg.bak; cp -R /newrepo/.hg /repo/.hg. No need to pull/push indeed. – Cartridge

The killall -s2 hg did the trick for me. Thanks! Btw, it seems I have to do it every time I change the .hgignore file. – Leatherleaf

If you want to stop the inotify servers from automatically running thus requiring you to kill them every time you update .hgignore, just put the line: hgext.inotify = ! in the [extensions] section of your .hgrc file. The ! tells Mercurial not to start the inotify extension/server. – Suffuse

Answer 2

Are both users running the same version of hg? Do they both have permission to read the .hgignore file?

Easement

Absolutely, only one version of Mercurial is installed. Just re-checked. I even tried making the second user the owner of .hgignore -- still no success. – Cartridge

Answer 3

Hrm. Just to be sure, what command is telling you they're parsed for some users and ignored from others? Are you confirming that with hg status --ignored? Somtimes people forget that 'hg add' overrides ignored file status, so if one user has already added (and maybe committed) those files but not pushed that add to the repo the other is viewing then that person can easily think their hgignore isn't being consulted when really it's just that added files make ignore irrelevant.

Were it me I'd move on to using strace next. Something like strace hg status --ignored | grep hgignore and compare the output for two users.

Spirelet

I was checking if .hgignore is read by adding a line that would cause a syntax error. Anyway, I've added a strace output, see above (BTW, strace is writing to stderr, so simply | grep will not work). Strange thing is, .hgignore is read on hg status --ignore and not on hg status, for one user. – Cartridge

Answer 4

Mercurial is written using python, henceforth finding out what happens should not be too hard. You just put traces in mercurial/ignore.py (simply use the warn() function, it's purpose is to print warnings to console from mercurial commands). If you put warn("I'm here") just before of pat = {} in the function def ignore(root, files, warn) it should print the warning "I'm here" each time it takes into account the .hgignore. After that, it's up to you to ask for the right traces to understand the behavior. If you know python, it should not take more than a few minutes.

Ignorant

Looks like this ignore module is sometimes just not getting loaded. Currently trying to debug mercurial/demandimport.py instead... – Cartridge

maybe it's some problem with sys.path in user context ? Or is your user running python interpreter in a sandbox or a different version of python, or something like that ? – Ignorant

Answer 5

Grea^W Some success.

There was another hg repository hiding in one of the subfolders, this seems the most probable cause of the problems described. I guess the another .hgignore was read before the one I actually needed, and all the following .hgignore-s were dropped.

However, moving that repository out and leaving a symlink on its place did not help at once.

Then, I decided to act radically:

• Deleted a cookie from ~/.hgcookies (I guess created by Kiln authentication module)
• Removed /project/.hgignore at all
• Removed symlink to other repository (described above)
• Added [ui] ignore = .hgignore to ~/.hgrc
• Committed/pushed changes
• Re-created /project/.hgignore, added/committed/pushed again
• Added symlink -- now it is ignored for either of users

Not sure which step exactly did the trick, but everything seems working as expected now, /project/.hgignore is parsed for every user.

[ui] ignore = .hgignore is not the solution as I removed it later and it did not break anything.

So, the problem is solved but WTF is still unanswered (:

Thank you all. And yes, do not rely on nested repositories.

Cartridge

"do not rely on nested repositories" is the wrong conclusion. Just make sure that you tell mercurial about any nested repositories in the versioned .hgsub file, as described here. – Easement