Windows CRLF to Unix LF Issues in Vagrant

Question

I'm working on provisioning a few VMs using Vagrant. Here's the situation:

Host: Windows 7 (64-bit)

Guest: Ubuntu 14.04 (64-bit)

I am having an issue getting the CRLF line endings to convert to LFs. This is causing the bash scripts in the shared folder to fail within the guest machine (see below).

vagrant@vagrant-host:/vagrant/bin$ sudo bash build-ubuntu-14.04.1-c
make.sh
build-ubuntu-14.04.1-cmake.sh: line 5: $'\r': command not found
build-ubuntu-14.04.1-cmake.sh: line 19: $'\r': command not found
: invalid option04.1-cmake.sh: line 21: set: -
set: usage: set [-abefhkmnptuvxBCHP] [-o option-name] [--] [arg ...]
build-ubuntu-14.04.1-cmake.sh: line 22: $'\r': command not found
build-ubuntu-14.04.1-cmake.sh: line 24: $'\r': command not found
build-ubuntu-14.04.1-cmake.sh: line 26: $'\r': command not found
build-ubuntu-14.04.1-cmake.sh: line 29: $'\r': command not found
build-ubuntu-14.04.1-cmake.sh: line 36: $'\r': command not found
build-ubuntu-14.04.1-cmake.sh: line 42: $'\r': command not found
build-ubuntu-14.04.1-cmake.sh: line 46: $'\r': command not found
build-ubuntu-14.04.1-cmake.sh: line 48: $'\r': command not found
build-ubuntu-14.04.1-cmake.sh: line 50: $'\r': command not found
build-ubuntu-14.04.1-cmake.sh: line 226: syntax error: unexpected end of file

In my Vagrantfile I have set the shell privisioner parameter binary to false.

# Provision the VM
ubuntu.vm.provision "shell" do |s|
  # replace Windows line endings with Unix line endings
  s.binary = false

  s.inline = "sudo apt-get update;
              sudo bash vagrant/bin/build-ubuntu-14.04.1-cmake.sh"
end

As per the Vagrant documentation:

binary (boolean) - Vagrant automatically replaces Windows line endings with Unix line endings. If this is true, then Vagrant will not do this. By default this is "false". If the shell provisioner is communicating over WinRM, this defaults to "true".

What's the issue here? Am I overlooking something in the documentation?

---

Update 1: I have tried to edit my local Git settings as recommended in this Stack Overflow answer, but no luck. Also, I have added a .gitattributes file to the root directory of the project and added the following to that file:

# detect all text files and automatically normalize them (convert CRLF to LF)
*       text=auto

I have also read over "Dealing with line endings" document provided by Git. When I commit to my repository CRLFs are converted to LFs, but when I checkout changes in a Windows workspace the LFs are converted to CRLFs. This is the exact behavior I want in my Git workflow. The issue is with Vagrant. The binary flag that I set doesn't perform the way the documentation describes.

---

Update 2: Changing s.binary = true fixed the issue. However, I think the wording in the documentation should be re-addressed. The documentation states that "If this [the flag] is true, then Vagrant will not do this [change CRLF to LF]." As I understand then, Vagrant will not change CRLFs to LFs if this flag is set. However, CRLFs are changed to LFs if this is set to true.

Kassey

You should answer your own question, since you solved it. Also, git's line ending stuff only has an effect on data moving between the filesystem and the git database. It won't do a recursive dos2unix on your shared folder. It would be effective if you had a separate working directory for windows and for linux, and moved code between them with git pull, though. You could have the canonical committed version with LF endings, and the Windows checkout with CRLF line endings. (and the Linux checkout with no conversion, or with forced-to-LF endings.) – Gudrun

Which vagrant version were you using at the time? – Intermission

@DeanRather I was using the latest, 1.7.1 – Kassey

Answer 1

As was specified above in my update, changing s.binary = true fixed the issue. However, I think the wording in the documentation should be re-addressed. The documentation states that "If this [the flag] is true, then Vagrant will not do this [change CRLF to LF]." As I understand then, Vagrant will not change CRLFs to LFs if this flag is set. However, CRLFs are changed to LFs if this is set to true.

Kassey

It seems that they have corrected the documentation now: vagrantup.com/docs/provisioning/shell#binary – Crutchfield

Answer 2

You're right, the documentation about binary was misleading. I've proposed a pull-request and this has been corrected on the documentation page.

So now it states:

binary (boolean) - Vagrant automatically replaces Windows line endings with Unix line endings. If this is false, then Vagrant will not do this. By default this is false. If the shell provisioner is communicating over WinRM, this defaults to true.

So to replace Windows line endings (CRLF) with Unix line endings (LF) you need to set:

s.binary = true

---

Alternative solutions includes:

• Changing line endings manually by:

  • using dos2unix command,

  • using ex command, e.g.

    ex +'bufdo! %! tr -d \\r' -scxa *.sh
    

• Add the following lines into your Bashrc file (e.g. ~/.bashrc)^gist:

  export SHELLOPTS
  set -o igncr
  

If you're using Git for versioning your code, you should:

• Configure Git on OS X to properly handle line endings by setting core.autocrlf option to input or false.

  If you've installed Git On Windows, the most common mistake is to select Checkout Windows-style option during installation, so you should re-install it and choose either: Checkout as-is and commit Unix-style line endings (core.autocrlf is set to input) or Checkout as-is, commit as-is (core.autocrlf is set to false).

• Consider creating git normalization file in your repository (.gitattributes) which ensures no CRLF line-endings are in place, neither on checkout nor on checkin, for example:

  *.sh     text eol=lf
  

  So people editing your provisioning script, they won't break the line endings.

• Read also: Dealing with line endings at GitHub Help.

• Related: '\r': command not found - .bashrc / .bash_profile.

Lorin

This is great news! Thanks. – Kassey

Answer 3

Perhaps completing the above:

September 2020, using Vagrant 2.2.10 on a Windows 10 computer, the default behaviour apparently is to try and replace Windows line endings. This however means that if you do use Linux line endings in your Vagrantfile - it will not work!

I was perplexed by getting several slightly different errors like

"Vagrantfile:1: syntax error, unexpected tIDENTIFIER, 
expecting end-of-input [+ reference to different places in the Vagrantfile]"

By guesswork I finally got the idea to change my line endings FROM Linux LF TO Windows CR+LF, which made it all work.

TL;DR: Using Vagrant on windows, use Windows line endings (CR+LF) in your Vagrantfiles.

Florie