git and CR vs LF (but NOT CRLF)

Question

This may sound like a redundant question (and may very well be a redundnant question) but I can't find the answer. Here's the situation:

My application is creating text files that have CR's as line endings. More specifically I'm not explicitly setting the line endings to CR, it just happens to be the output of the command I'm using to get the text body. Of course I could manually convert the CR's to LF's but I don't want to if I can avoid it.

Git's treating these files as a single line (e.g. during a diff). I have determined via this test repository that the line-endings are the cause: https://github.com/jfletcher4d/diff-test

I don't actually care what the line-endings are on the file system. It's not important, at least not yet (I may eventually care if I need to import these files, right now it's only export). But I do not want to convert the CR's to LFs in my application if I can avoid it, for performance reasons as well as anal-retentativeness reasons :) I.e. this isn't a question of how to create text files but, rather, how to force every text file in the repo to have only LFs.

Can git be configured to change all line endings to LF, even if the files are committed with CR in them?

I'm on Windows, using TortoiseGit and msysgit (and a little TortoiseSVN on the side) interchangeably while I learn git, if that matters.

Dislimn

This is troubling: “But do we want to support CR-only? Probably not.” github.com/git/git/blob/master/convert.c#L93 – Damicke

And: “We're currently not going to even try to convert stuff that has bare CR characters. Does anybody do that crazy stuff?” – Damicke

possible duplicate of How do I force git to use LF instead of CR+LF under windows? – Flax

@JoshLee Haha, fair enough. There's some "legacy" here I think; that's why I'm getting bare CR's in the text. So yes somebody is still doing this :) – Dislimn

Answer 1

Git doesn’t appear to support CR line endings, so I would write a filter to convert the newlines. The files in the work tree will have CR line endings, and they will be transparently converted to LF when they are indexed. The filter has two parts: “clean” checks the files in, and “smudge” checks the files out.

Use this in .git/config:

[filter "cr"]
    clean = tr '\\r' '\\n'
    smudge = tr '\\n' '\\r'

And .git/info/attributes (or .gitattributes if it should be versioned)

* filter=cr

Note that this automatically makes git-diff happy, since it will use the “clean” version.

Just remember to set the pattern to just the files you need, or binary files will be corrupted and all text files will check out with CR line endings.

Also note that if the filter is not configured, it will silently fail, so add the config line when setting up a new copy of the repository.

Damicke

Makes sense. However, would someone cloning this repo get LF's on their system? I.e. does this only work because I started with CR's in the first place? I probably need to understand "clean" vs "smudge" so bear with me, I'm new to this. Edit: I'll test this tomorrow and mark accepted if it works ok. – Dislimn

Correct. In order for this to work after a clone, the attributes should be versioned, and the user should have the filters set up in .gitconfig in their home directory. (Or in /etc — see git-config(1)) – Damicke

Got it, thanks. I'll give this a whirl (stuck in meetings today, unfortunately). – Dislimn

Finally have a chance to work on this and running into a couple of issues. 1) is there any way to version the actual filter so that all users have it? I.e. must the user set up the filter? 2) Can the attribute pattern apply recursively? I don't want to filter all txt files, only txt files in a certain folder (and subfolders). Thanks! – Dislimn

Looks like it might be a bug/doc bug? code.google.com/p/msysgit/issues/detail?id=332 – Dislimn

I can confirm that by moving .gitattributes into the root of the folder tree I care about and using "*.txt" for the pattern, instead of using "folder/*.txt" (or "folder/*") as a pattern, it does apply the filter recursively. Looks like folder patterns don't work. – Dislimn

Answer 2

FWIW, I ended up converting the line endings to LF on the local side of things. It ended up being the simpler/less buggy solution.

Dislimn