Mercurial hg status displaying ignored files

Question

Can anyone tell me why mercurial is displaying ignored files for the 'hg status' command?

Here's the confirmation that mercurial is indeed ignoring the files:

$ hg addremove
$ hg commit 
nothing changed     

Now, have a look at the status output. I ran this immediately after the lines above. (Prinout is partial, with identifiable text removed)

$ hg status
? Core/target/Core-0.0.1-SNAPSHOT-tests.jar
? Core/target/Core-0.0.1-SNAPSHOT.jar
? Core/target/classes/META-INF/MANIFEST.MF
? Core/target/classes/xxx/yyy/zzz/X.class
? Core/target/classes/xxx/yyy/zzz/XBackup.class
? Core/target/classes/xxx/yyy/zzz/XBackupInput.class
? Core/target/classes/xxx/yyy/zzz/XBackupOutput.class
? Core/target/classes/xxx/yyy/zzz/XImpl$GetResultsAndStatistics.class
? Core/target/classes/xxx/yyy/zzz/XImpl$MonitoringMode.class
? Core/target/classes/xxx/yyy/zzz/XImpl$UpdateMode.class
? Core/target/classes/xxx/yyy/zzz/XImpl.class
? Core/target/classes/xxx/yyy/zzz/XIsFullException.class
? Core/target/classes/xxx/yyy/zzz/XSource.class
? Core/target/classes/xxx/yyy/zzz/XUsageException.class
? Core/target/classes/xxx/yyy/zzz/CheckResults.class
? Core/target/classes/xxx/yyy/zzz/Noise.class
? Core/target/classes/xxx/yyy/zzz/core/DateTimeGenerator$TemporalMeasure.class
? Core/target/classes/xxx/yyy/zzz/core/DateTimeGenerator.class
? Core/target/classes/xxx/yyy/zzz/core/LoggingConfigurator.class
? Core/target/classes/xxx/yyy/zzz/core/ManagedIterator.class
? Core/target/classes/xxx/yyy/zzz/core/NetworkUtils.class
? Core/target/classes/xxx/yyy/zzz/core/StackTraceUtils.class
? Core/target/classes/xxx/yyy/zzz/core/Summarisable.class
? Core/target/classes/xxx/yyy/zzz/core/TimingUtils.class
? Core/target/classes/xxx/yyy/zzz/core/XMLStaticStringUtils.class
? Core/target/classes/xxx/yyy/zzz/core/Zipper.class
...

There's something like a thousand lines of these ignored files showing in the status printout; that's some serious guff.

How can I get rid of this?

Thanks

UPDATE:

Here's my .hgignore file.

syntax:glob
.metadata*

syntax:regexp
^target.*$
^[^/]*/target.*$
^[^/]*/[^/]*/target.*$
^bin.*$
^[^/]*/bin.*$
^[^/]*/[^/]*/bin.*$

UPDATE:

Modified regexps a wee bit * -> .*

Teacake

Strange. Which version of Mercurial are you using? How does your .hgignore file look like? Any useful information from hg verify? – Cavitation

@Oben version is 1.8. .hgignore file has been added to the original post. And hg verif ydoesn't spit out anything much beyond 'checking...' lines and the number of files and changesets – Teacake

May-be something strange in your configuration: what's the output of "hg showconfig"? – Digitalin

If you're using a recent version of Mercurial you can try the hg debugignore command. What does that output look like? Are you sure you're not using the inotify extension? Some distributions turn that on by default and really shouldn't. – Electrothermal

I'm afraid it's working now thanks to the multiple reverts I mentioned in another answer. I can't reproduce the issue any more. – Teacake

Answer 1

Many thanks for everyone's help on this. I haven't found an answer to the problem but I did find a workaround that got rid of it. So I'm assuming that at some point when modifying the hgignore regexps mercurial got confused and corrupted it's internal data... or something. I don't know enough internals to know what this could have been.

Anyway, should you find yourself in the same boat, here's what I did:

1. Make sure your .hgignore file says what you want it to and that all the files turning up in the hg status are the ones you want to ignore.
2. Add all the files to be ignored. hg addremove doesn't work because it does honour the hgignore contents, so I used Eclipse's hg plugin and did each file manually (well, I selected them all and Eclipse applied the add to them all individually).

3. Revert all added files. I used the following because I'm on linux. (thanks to another poster on a separate question here on SO - which I can now no longer find unfortunately)

   hg status -an0 | xargs -0 hg revert

4. repeat the above step until it no longer works. In my case this was sometimes up to as many as 3 reverts in a row! (No idea what it was actually doing though :( )

5. commit

You should now no longer see the ignored files in the hg status output.

I'm sorry, I have no idea why this worked, or what the original problem was. If anyone does, I'd be very interested to know your thoughts.

Many thanks again to everyone :)

Teacake

Answer 2

Your regexp syntax is wrong. You have this:

syntax:regexp
^target*$

which means "ignore anything beginning with target and ending with an asterisk

Which fails to ignore these:

Core/target/classes/META-INF/MANIFEST.MF
Core/target/classes/xxx/yyy/zzz/X.class

for two reasons -- they begin with Core/ not target and they don't end with an asterisk.

What you probably meant was this:

syntax:regexp
^.*/target/.*$

which matches anything that has /target/ in it (notice it's .* in a regexp * is for glob style). However the ^ and $ only serve to root your regex and you don't want it rooted -- you want to find it anywhere in the string, so just do this:

syntax:regexp
/target/

The clue in all this was that the files were marked with ? which means not ignored and not added, if they were ignored you wouldn't see them at all without adding --ignored to the status command.

Electrothermal

You're right that there is an error in that regexp, but I'm afraid you're wrong in the details. That regexp is supposed to catch ONLY the top-level directory 'target' and it's children. The "" should be a ".". The next two regexps then repeat this sequence for all second-level 'target' directories and then all third-level 'target' directories. For some unfortunate and lamentable reasons we project-specific requirements to allow target directories below this level. – Teacake

Also, rather than just bounce back about the regexps, I feel compelled to point out that the workaround solution I documented in my answer, plus the original fact mercurial does correctly enforce the hgignore file as expected in all but the status command strongly suggests it really isn't a problem with the regexps. – Teacake

You never said what you were trying to ignore and since the regexps didn't do what you thought everyone had to guess. This is one of those cases where asking "how do I do what I want, and what I want is X" would be better than "what is wrong with this?" Your regexps are way overwrought for what you're trying to do. Be clear about that and we'll get this down to 2 lines I wager. – Electrothermal

I have to agree with Uberpuppy. While the ignore expressions are a bit curious they work for the example files given in the question and obviously they are supposed to ignore target and bin directories only at certain levels. The main issue is why hg st shows the files as untracked while hg addremove appears to ignore them. – Cavitation

In my case - ignore all files in subdirectories called "target" - the regexp ^.*/target/.*$ was the solution. Thanks! – Hawkins

Answer 3

I got similar issue because I changed letter case for some files. So previously I got file called "sitenav.ext", renamed it to "siteNav.ext" and started to have same sort of issues - when I tried to "hg st" I got "? siteNav.ext". Fix was easy rename to "_siteNav.ext", addremove, commit, rename back to "siteNav.ext", addremove, commit -> profit!

Smriti

Answer 4

As commented by Jerome in the question, providing your config might help.

One (admittedly strange) reason for your problem could be a permission issue with the .hgignore file combined with exclude defaults for the addremove command.

I could reproduce your situation by revoking any read permission from the ignore file and by using -X "**/target/**" as a default option for the addremove command (which might be set in any of the possible Mercurial configuration files):

$ hg st
# no output, as expected
$ chmod 000 .hgignore
$ hg addremove # with '-X "**/target/**"' set as default somewhere
$ hg commit 
nothing changed  
$ hg st
? Core/target/Core-0.0.1-SNAPSHOT-tests.jar
? Core/target/Core-0.0.1-SNAPSHOT.jar
...

hg fails in reading the ignore file and thus does not know that the target stuff should be ignored. A corresponding warning from hg would be helpful here.

This question has been updated in response to Jerome's comment.

Cavitation

In that situation, wouldn't "hg addremove" add all files? – Digitalin

@Jerome: Er, yes, you're right. I think I should revoke my answer. – Cavitation

@Jerome: I did not revoke the answer but updated it to a more specific case that also considers addremove. – Cavitation

Yeah, I expected this :) Indeed my answer describes a really exceptional case. – Cavitation

Answer 5

? does not mean ignored.

It just means that mercurial is not tracking the file.

If you don't want to see them just do

hg status -mard

i.e. only show modified, added, removed, deleted

Nightly

I think the poster is aware of this, but wonders why they are listed as untracked, even though they should either be ignored or tracked after hg addreomve; hg commit has been issued. – Cavitation

Euch. That works, but there's two problems with it. 1) Both I and all my team members have to type that extra every time we run status for ever more. That's lame. More problematically, 2) IDE integration or other automated processes that run just hg status will get spurious outputs. – Teacake

@Oben Precisely. It seems to be inconsistent with the stated documentation, for example: mercurial.selenic.com/wiki/… – Teacake

Sorry for the missunderstanding – Nightly

Answer 6

Your .hgignore file says "^target" is ignored, but the files are in Core/target. So the ignore line should rather be "target" (without the ^) or "^Core/target".

Or am I missing sth here?

(Note: ^ implies that regex matching starts from the beginning of the path, i.e. only paths starting with target would be matched!)

Edmonson

Answer 7

Had the same problem and Bob's answer did the trick for me.

If you are using TortoiseHg you can select "edit ignore filter" in the context menu (windows) to verify and change your .hgignore settings there.

Cobnut

Welcome to Stack Overflow! When your addition is just a small comment, you can also directly comment on (Bob's) answer. He may then incorporate the feedback into his answer. – Thermoelectrometer