How do I remove leading whitespace chars from Ruby HEREDOC?

Question

I'm having a problem with a Ruby heredoc i'm trying to make. It's returning the leading whitespace from each line even though i'm including the - operator, which is supposed to suppress all leading whitespace characters. my method looks like this:

    def distinct_count
    <<-EOF
        \tSELECT
        \t CAST('#{name}' AS VARCHAR(30)) as COLUMN_NAME
        \t,COUNT(DISTINCT #{name}) AS DISTINCT_COUNT
        \tFROM #{table.call}
    EOF
end

and my output looks like this:

    => "            \tSELECT\n            \t CAST('SRC_ACCT_NUM' AS VARCHAR(30)) as
COLUMN_NAME\n            \t,COUNT(DISTINCT SRC_ACCT_NUM) AS DISTINCT_COUNT\n
        \tFROM UD461.MGMT_REPORT_HNB\n"

this, of course, is right in this specific instance, except for all the spaces between the first " and \t. does anyone know what i'm doing wrong here?

Templin

Answer 1

The <<- form of heredoc only ignores leading whitespace for the end delimiter.

With Ruby 2.3 and later you can use a squiggly heredoc (<<~) to suppress the leading whitespace of content lines:

def test
  <<~END
    First content line.
      Two spaces here.
    No space here.
  END
end

test
# => "First content line.\n  Two spaces here.\nNo space here.\n"

From the Ruby literals documentation:

The indentation of the least-indented line will be removed from each line of the content. Note that empty lines and lines consisting solely of literal tabs and spaces will be ignored for the purposes of determining indentation, but escaped tabs and spaces are considered non-indentation characters.

Earwitness

I love that this is still a relevant subject 5 years after I asked the question. thanks for the updated response! – Templin

@ChrisDrappier Not sure if this is possible, but I'd suggest to change the accepted answer for this question to this one as nowadays this clearly is the solution. – Jackstraw

Answer 2

If you can't use Ruby 2.3 or newer, but do have Rails 3.0 or newer, try #strip_heredoc. This example from the docs prints the first three lines with no indentation, while retaining the last two lines' two-space indentation:

if options[:usage]
  puts <<-USAGE.strip_heredoc
    This command does such and such.
 
    Supported options are:
      -h         This message
      ...
  USAGE
end

The documentation also notes: "Technically, it looks for the least indented line in the whole string, and removes that amount of leading whitespace."

Here was its Rails 3-era implementation from active_support/core_ext/string/strip.rb:

class String
  def strip_heredoc
    indent = scan(/^[ \t]*(?=\S)/).min.try(:size) || 0
    gsub(/^[ \t]{#{indent}}/, '')
  end
end

And you can find the matching tests in this version of test/core_ext/string_ext_test.rb.

Roybn

You can still use this outside of Rails 3! – Sumac

iconoclast is correct; just require "active_support/core_ext/string" first – Crossfertilize

Doesn't seem to work in ruby 1.8.7: try is not defined for String. In fact, it seems that it is a rails-specific construct – Anitraaniweta

Answer 3

Not much to do that I know of I'm afraid. I usually do:

def distinct_count
    <<-EOF.gsub /^\s+/, ""
        \tSELECT
        \t CAST('#{name}' AS VARCHAR(30)) as COLUMN_NAME
        \t,COUNT(DISTINCT #{name}) AS DISTINCT_COUNT
        \tFROM #{table.call}
    EOF
end

That works but is a bit of a hack.

EDIT: Taking inspiration from Rene Saarsoo below, I'd suggest something like this instead:

class String
  def unindent 
    gsub(/^#{scan(/^\s*/).min_by{|l|l.length}}/, "")
  end
end

def distinct_count
    <<-EOF.unindent
        \tSELECT
        \t CAST('#{name}' AS VARCHAR(30)) as COLUMN_NAME
        \t,COUNT(DISTINCT #{name}) AS DISTINCT_COUNT
        \tFROM #{table.call}
    EOF
end

This version should handle when the first line is not the one farthest to the left too.

Ataractic

I feel dirty for asking, but what about hacking the default behavior of EOF itself, rather than just String? – Goering

Surely the behaviour of EOF is determined during parsing, so I think what you, @patcon, are suggesting would involve changing the source code for Ruby itself, and then your code would behave differently on other versions of Ruby. – Ataractic

I kinda wish Ruby's dash HEREDOC syntax worked more like that in bash, then we wouldn't have this problem! (See this bash example ) – Obtuse

Pro-tip: try either of these with blank lines in the content and then remember that \s includes newlines. – Everyplace

I tried that on ruby 2.2 and didn't notice any problem. What happend for you? (repl.it/B09p) – Ataractic

@juanpastas Great! – Ataractic

Answer 4

Here's a far simpler version of the unindent script that I use:

class String
  # Strip leading whitespace from each line that is the same as the 
  # amount of whitespace on the first line of the string.
  # Leaves _additional_ indentation on later lines intact.
  def unindent
    gsub /^#{self[/\A[ \t]*/]}/, ''
  end
end

Use it like so:

foo = {
  bar: <<-ENDBAR.unindent
    My multiline
      and indented
        content here
    Yay!
  ENDBAR
}
#=> {:bar=>"My multiline\n  and indented\n    content here\nYay!"}

---

If the first line may be indented more than others, and want (like Rails) to unindent based on the least-indented line, you may instead wish to use:

class String
  # Strip leading whitespace from each line that is the same as the 
  # amount of whitespace on the least-indented line of the string.
  def strip_indent
    if mindent=scan(/^[ \t]+/).min_by(&:length)
      gsub /^#{mindent}/, ''
    end
  end
end

Note that if you scan for \s+ instead of [ \t]+ you may end up stripping newlines from your heredoc instead of leading whitespace. Not desirable!

Everyplace

Answer 5

<<- in Ruby will only ignore leading space for the ending delimiter, allowing it to be properly indented. It does not strip leading space on lines inside the string, despite what some documentation online might say.

You can strip leading whitespace yourself by using gsub:

<<-EOF.gsub /^\s*/, ''
    \tSELECT
    \t CAST('#{name}' AS VARCHAR(30)) as COLUMN_NAME
    \t,COUNT(DISTINCT #{name}) AS DISTINCT_COUNT
    \tFROM #{table.call}
EOF

Or if you just want to strip spaces, leaving the tabs:

<<-EOF.gsub /^ */, ''
    \tSELECT
    \t CAST('#{name}' AS VARCHAR(30)) as COLUMN_NAME
    \t,COUNT(DISTINCT #{name}) AS DISTINCT_COUNT
    \tFROM #{table.call}
EOF

Beautiful

-1 For stripping all leading whitespace instead of just the indentation amount. – Everyplace

@Everyplace The OP mentioned that he expected it to "suppress all leading whitespace characters," so I gave an answer that did that, as well as one that only stripped the spaces, not the tabs, in case that's what he was looking for. Coming in several months later, downvoting answers that worked for the OP, and posting your own competing answer is kind of lame. – Beautiful

@BrianCampbell I'm sorry you feel that way; no offense was intended. I hope you believe me when I say that I'm not downvoting in an attempt to garner votes for my own answer, but simply because I came upon this question through an honest search for similar functionality and found the answers here sub-optimal. You are right that it solves the OP's exact need, but so does a slightly-more-general solution that provides more functionality. I also hope you would agree that answers posted after one has been accepted are still valuable to the site as a whole, particularly if they offer improvements. – Everyplace

Finally, I wanted to address the phrase "competing answer". Neither you nor I should be in competition, nor do I believe that we are. (Though if we are, you're winning with 27.4k rep as of this moment. :) We help individuals with problems, both personally (the OP) and anonymously (those arriving via Google). More (valid) answers help. In that vein, I reconsider my downvote. You are right that your answer was not harmful, misleading, or overrated. I have now edited your question just so that I could bestowed the 2 points of rep I took away from you. – Everyplace

@Everyplace Sorry about being grumpy; I tend to have a problem with "-1 for something I don't like" replies for answers that adequately address the OP. When there are already upvoted or accepted answers which almost, but not quite, do what you want, it tends to be more helpful for anyone in the future to just clarify how you think the answer could be better in a comment, rather than downvoting and posting a separate answer which will show up far below and won't usually be seen by anyone else who has the problem. I only downvote if the answer is actually wrong or misleading. – Beautiful

@BrianCampbell I agree that the downvote was incorrect. Thanks for the reminder to enforce my own internal criteria for downvoting (which are the same as yours, but also include massively overrated answers). – Everyplace

Answer 6

Some other answers find the indentation level of the least indented line, and delete that from all lines, but considering the nature of indentation in programming (that the first line is the least indented), I think you should look for the indentation level of the first line.

class String
  def unindent; gsub(/^#{match(/^\s+/)}/, "") end
end

Philia

Psst: what if the first line is blank? – Everyplace

Answer 7

Like the original poster, I too discovered the <<-HEREDOC syntax and was pretty damn disappointed that it didn't behave as I thought it should behave.

But instead of littering my code with gsub-s I extended the String class:

class String
  # Removes beginning-whitespace from each line of a string.
  # But only as many whitespace as the first line has.
  #
  # Ment to be used with heredoc strings like so:
  #
  # text = <<-EOS.unindent
  #   This line has no indentation
  #     This line has 2 spaces of indentation
  #   This line is also not indented
  # EOS
  #
  def unindent
    lines = []
    each_line {|ln| lines << ln }

    first_line_ws = lines[0].match(/^\s+/)[0]
    re = Regexp.new('^\s{0,' + first_line_ws.length.to_s + '}')

    lines.collect {|line| line.sub(re, "") }.join
  end
end

Sandie

+1 for the monkeypatch and stripping only the indenting whitespace, but -1 for an overly-complex implementation. – Everyplace

Agree with Phrogz, this really is the best answer conceptually, but the implementation is too complicated – Ataractic

Answer 8

Note: As @radiospiel pointed out, String#squish is only available in the ActiveSupport context.

---

I believe ruby's String#squish is closer to what you're really looking for:

Here is how I would handle your example:

def distinct_count
  <<-SQL.squish
    SELECT
      CAST('#{name}' AS VARCHAR(30)) as COLUMN_NAME,
      COUNT(DISTINCT #{name}) AS DISTINCT_COUNT
      FROM #{table.call}
  SQL
end

Baskett

Thanks for the down vote, but I believe we would all better benefit from a comment that would explain why this solution should be avoided. – Baskett

Just a guess, but String#squish is probably not part of ruby proper, but of Rails; i.e. it won't work unless using active_support. – Unpen

Thanks, that's perfect if you want to break a long paragraph of text on multiple lines for better readability in the code, but in the same time treat the line breaks as empty spaces. – Yean

Answer 9

another easy to remember option is to use unindent gem

require 'unindent'

p <<-end.unindent
    hello
      world
  end
# => "hello\n  world\n"  

Chancelor

Answer 10

I needed to use something with system whereby I could split long sed commands across lines and then remove indentation AND newlines...

def update_makefile(build_path, version, sha1)
  system <<-CMD.strip_heredoc(true)
    \\sed -i".bak"
    -e "s/GIT_VERSION[\ ]*:=.*/GIT_VERSION := 20171-2342/g"
    -e "s/GIT_VERSION_SHA1[\ ]:=.*/GIT_VERSION_SHA1 := 2342/g"
    "/tmp/Makefile"
  CMD
end

So I came up with this:

class ::String
  def strip_heredoc(compress = false)
    stripped = gsub(/^#{scan(/^\s*/).min_by(&:length)}/, "")
    compress ? stripped.gsub(/\n/," ").chop : stripped
  end
end

Default behavior is to not strip newlines, just like all the other examples.

Glucinum

Answer 11

I collect answers and got this:

class Match < ActiveRecord::Base
  has_one :invitation
  scope :upcoming, -> do
    joins(:invitation)
    .where(<<-SQL_QUERY.strip_heredoc, Date.current, Date.current).order('invitations.date ASC')
      CASE WHEN invitations.autogenerated_for_round IS NULL THEN invitations.date >= ?
      ELSE (invitations.round_end_time >= ? AND match_plays.winner_id IS NULL) END
    SQL_QUERY
  end
end

It generates excellent SQL and do not go out of AR scopes.

Disbelieve