invalid multibyte char (US-ASCII) with Rails and Ruby 1.9

Question

I'm using Ruby 1.9.1 with Rails 2.3.4 My application is to handle text input

If I try something like (the inside quotation marks look different)

text = "”“"

I get the following error:

#<SyntaxError: /Users/tammam56/rubydev/favquote/lib/daemons/twitter_quotes_fetch.rb:54: invalid multibyte char (US-ASCII)
/Users/tammam56/rubydev/favquote/lib/daemons/twitter_quotes_fetch.rb:54: invalid multibyte char (US-ASCII)
/Users/tammam56/rubydev/favquote/lib/daemons/twitter_quotes_fetch.rb:54: syntax error, unexpected $end, expecting keyword_end

I need to user those quotation marks as users might input them and I have to account for that?

Any ideas?

Gnaw

If your code does not have any backticks in it but you're being "accused" of using backticks, there may be some weird spacing/tabs/newline issues in your file. Try posting it into a StackOverflow blank for example, and SO will start acting weird. Remove the strange spaces and tabs and newlines. Again, just pasting the code into a SO blank and trying to format your code for presentation is one way to give yourself a hint. – Sanatory

Answer 1

Have you tried adding a magic comment in the script where you use non-ASCII chars? It should go on top of the script.

#!/bin/env ruby
# encoding: utf-8

It worked for me like a charm.

Ladawnladd

Hmm.........added that to the top of the file but still get the same error message. Any suggestions? – Bowshot

@ArtemKalinchuk Just add it directly under #!/bin/env ruby I had a blank line between them and it didn't work. – Zitella

The central explanation can be found in the article @dalyons linked: source files receive a US-ASCII Encoding, unless you say otherwise. If you place any non-ASCII content in a String literal without changing the source Encoding, Ruby will die with that error. Thanks guys, I finally got it :-) – Telstar

Awesome! How to set this for all files of my Rails project? – Bianca

I'm sure you can come up with a series of pipelined bash instructions for adding this to all .rb, .erb, .yml, etc.. files in a RoR folder structure. – Urinalysis

#!/bin/env ruby isn't necessary unless you're running the script from the command line as an executable. The # encoding line works by itself. – Edrick

make sure after setting that magic comment that your file is encoded in utf-8 – Catawba

# encoding: utf-8. It's late 2013 and we still have to play this game. Hold on, phone ringing... It was 2033, they called to say they still play it. Oh well, thanks for reminding me, Jarek Zmudzinski from 2010. – Lannielanning

@gotqn - Please find the same article here - graysoftinc.com/character-encodings/… – Annoy

This worked for me on Ubuntu 14.04, running rails 3.2.11 – Melville

Answer 2

If you want to add magic comments on all the source files of a project easily, you can use the magic_encoding gem

sudo gem install magic_encoding

then just call magic_encoding in the terminal from the root of your app.

Kinsey

I think it's important to remember this kind of details, so I wouldn't use that gem for at least a few months of writing # encoding: utf-8 manually. – Urinalysis

adding 'gem magic_encoding' to gemfile on rails 2.3 & ruby 1.9 helped – Olomouc

this doesn't integrate into cucumber tests. – Rizas

@Olomouc You should never put external libraries to your project's Gemfile like that. magic_encoding is just a command-line tool, not a project dependency. – Irrelevancy

Answer 3

I just want to add my solution:

I use german umlauts like ö, ü, ä and got the same error.
@Jarek Zmudzinski just told you how it works, but here is mine:

Add this code to the top of your Controller: # encoding: UTF-8
(for example to use flash message with umlauts)

example of my Controller:

# encoding: UTF-8
class UserController < ApplicationController

Now you can use ö, ä ,ü, ß, "", etc.

Misbelief

Answer 4

That worked for me:

$ export LC_ALL=en_US.UTF-8
$ export LANG=en_US.UTF-8

Quiver

Answer 5

Those slanted double quotes are not ASCII characters. The error message is misleading about them being 'multi-byte'.

Sublingual

Why is it misleading? They are multibyte characters. – Phane

Because ASCII doesn't define any multi-byte encodings. As for as ASCII is concerned, those are gibberish, that happens to be valid in a related encoding. – Sublingual

Answer 6

Just a note that as of Ruby 2.0 there is no need to add # encoding: utf-8. UTF-8 is automatically detected.

Irrelevancy