git line endings : renormalize does not seem to checkout the right line endings

Question

I decided to set my line endings the Right Way via a .gitattributes file as detailed for instance here - so I set the core.autocrlf to false and created and committed a .gitattributes file :

*.java text eol=native
*.jsp text eol=native
*.css text eol=native
*.html text eol=native
*.js text eol=native
*.xml text eol=native
*.sql text eol=native
*.MF text eol=native

# git files
*.gitignore text eol=native
*.gitattributes text eol=native

#eclipse files
*.classpath text eol=native
*.project text eol=native
*.prefs text eol=native
*.properties text eol=native

I then issued git rm --cached -r . and then git reset --hard (tried also git checkout HEAD), as suggested here. Now all the files have LF line endings. Shouldn't be CRLF ? What do I miss ? I am on windows 7, git version 1.8.0.msysgit.0.

Thanks

Hooey

Did you try https://mcmap.net/q/11238/-trying-to-fix-line-endings-with-git-filter-branch-but-having-no-luck ? – Brewhouse

@Brewhouse : I issued rm .git/index then git reset but nothing changed - I had already committed the gitattributes - anyway the strange thing is the checkout should checkout windows line endings - what do I miss ? The gitattributes I wrote seems OK, no ? – Hooey

@Brewhouse : I rolled back and followed the proceeding to no avail - insists on checking out with unix line endings – Hooey

@Brewhouse : a bug apparently - see my answer below (and edit as you see fit - did not have time to report the bug properly) – Hooey

"did not have time to report the bug properly" and yet you wonder why it does not get fixed. There is your answer. Someone who actually reproduces the issue needs to spend the time generating tests and ensuring that it works under all circumstances. Without that input, it will remain broken under conditions that most windows users simply do not encounter. – Augustaaugustan

@patthoyts: I just saw you are related to git so I'll answer to your comment - I normally wouldn't cause I do not like being patronized. I did not say I won't report, I said I didnt. I did answer to a 2 year old thread in git's mailing list however, where the bug is reported as clearly as possible. Since my set up is far from uncommon I either suspect I have done something stupid (which becomes more unlikely after 2 days struggling) or that people simply do not care. This is not a marginal feature - just search SO for git autocrlf. If you can give some insight this would be really appreciated. – Hooey

@Mr_and_Mrs_D: Unfortunately, because the reply did not quote/include the original series, or at least a reference to the series on Gmane/148436, the report was rather too terse and did not read well. The MsysGit activity is desperately short of contributors and is in a frustrating Catch 22 situation. Without your positive contribution of a fix it will be just another 'one issue too many'. I don't believe Pat was being patronizing, rather he was just reflecting his frustrated reality. [I wish I was able to contribute more as well]. If only Windows had used LF. – Mange

@PhilipOakley : I do not understand what you refer to by : the report was rather too terse and did not read well. Please elaborate (maybe post an answer with details or at least some links?). As for the rest : I am rather novice to contribute fixes - moreover I have not even understood if this is a bug in git or in msysgit (?) – Hooey

@Mr_and_Mrs_D, It was that the email version (which is what I read) did not include any of the earlier (old) thread - It's different on the Nabble interface. This meant there was no initial context upon which to understand your situation. The final "Please fix" then felt too short and too directed (at the valiant few ;-). A softer question on the lines of "Does the (earlier) patch fix the the problem, and how can I help get it into the mainline?", or similar, would probably engage with the those who can best advise. Pat and his colleagues have seen too many unsupported "FixIt!" requests... – Mange

The other method is to simply suggest (with a patch) improvements to the documentation so that the right information is given in the right order. I know that the Windows line endings can be a real problem (My Q: stackoverflow.com/questions/6823168) – Mange

Another related question: .gitattributes with core.autocrlf unset – Balenciaga

@Brice:seems they are fixing it !!! HURRAY – Hooey

+1 for this question, I am encountering this :( – Nelidanelie

With Git 2.16 (Q1 2018), you will have git add --renormalize .: See [my other answer ](https://mcmap.net/q/11222/-git-how-to-renormalize-line-endings-in-all-files-in-all-revisions). – Brewhouse

Answer 1

It must be a bug. It is strange it is not fixed or reported really - this whole mess is about windows, and it does not work precisely on windows ? Moreover there is no mention of it anyplace (?)

EDIT : Installed from the mingw "latest repository catalogues" - g++, gcc, ObjC + the MinGW Developer Toolkit and MSYS-1.0.11. Same behavior. Whenever I try to commit a CRLF file I get the CRLF will be replaced with LF (on checkout is implied) warning.

EDIT 2 : seems about to be fixed

EDIT 3: This has been fixed in Git 1.8.4.

Hooey

+1 for the feedback. I will test on my side if I reproduce the issue. – Brewhouse

@VonC: thank you very much - to note that I am on mingwin (as detailed further down on the messages : comments.gmane.org/gmane.comp.version-control.git/148436) – Hooey

I don't know if it's helpful but I'm having some success by following the instructions here: When text=auto normalization is enabled in an existing repository, any text files containing CRLFs should be normalized. If they are not they will be normalized the next time someone tries to change them, causing unfortunate misattribution. From a clean working directory: ... – Nelidanelie

@ta.speot.is: thanks - I tried this also - see comment in my question – Hooey

Even though there was an attempt to fix this problem with eol=native on Windows recently, the fix was incomplete and the bug is still there. So, just don't use native EOL settings at the moment, they are broken :( – Wat

This bug has finally been fixed in Git for Windows 1.8.4. – Bartell

Answer 2

For what you are trying to do I think you need the following. Note that the eol attribute according to the gitattributes man page may be either "lf" or "crlf". Native is for the core.eol config setting.

Set core.autocrlf false to take explicit control of normalization. Now only files you mark with the text attribute will undergo normalization.

Either set the eol attribute explicitly to lf or crlf for specific filetypes (eg: shell scripts may require lf to work on unix, .csproj files may require crlf on windows). If the eol attribute is unset the value of core.eol should be used. If you set core.eol to crlf then your text files will get crlf endings.

Here is a git test script to illustrate this (run from git/t/):

#!/bin/sh
test_description='check native crlf setting'
. ./test-lib.sh

has_cr() {
        tr '\015' Q <"$1" | grep Q >/dev/null
}

test_expect_success 'test native elf' '
  printf "*.txt text\n" > .gitattributes
  printf "one\r\ntwo\r\nthree\r\n" > filedos.txt
  printf "one\ntwo\nthree\n" > fileunix.txt
  git init &&
  git config core.autocrlf false &&
  git config core.eol crlf &&
  git add . &&
  git commit -m "first" &&
  rm file*.txt &&
  git reset --hard HEAD &&
  has_cr filedos.txt && has_cr fileunix.txt
'

test_done

With the above config and attributes, with Git for Windows 1.8.0 both files are normalized after the reset and contain crlf line endings.

Where a bug may exist is that if the core.eol variable is left unset (or set to 'native') this test fails as the files are normalized to lf line endings in this case. The path you have mentioned above does not help this situation either. So from my testing, you must explicitly set core.eol to crlf to have your planned approach be successful.

Augustaaugustan

Thanks - I had put the eol=native in an attempt to force this. I know that setting eol=crlf would make checkouts use crlf and that some files might need either lf or crlf to work (none of the ones I posted though). But this is a full time bug : on windows native is CRLF - if I put this in my gitattributes it completely defeats its purpose - my linux collaborators would get CRLFs too ! The situation was much better with the --global core.autocrlf then !!! This must be fixed - I repeat checking out (via reset, checkout, clone etc) on windows should produce CRLF when core.eol=native – Hooey

Look carefully - if you set the attributes for these files just to 'text' and then configure your repository to use core.eol=crlf then your unix users will have their own core.eol setting (ie: lf) and it will work for everyone. core.eol = native does not currently work - this is a bug which i've now proposed a fix for and should be resolved in the next release. For now - 'git config core.eol crlf' and do not set eol in the attributes at all. – Augustaaugustan

Thanks - I got that - that's what I meant with the The situation was much better with the --global core.autocrlf then - I have to set a global preference as it is and have the gitattributes (while before it was only a global :). Please link to the bug report/fix you made in your answer so I can accept it. – Hooey