Getting "sed error - illegal byte sequence" (in bash) [duplicate]

Question

Doing some stream editing to change the nasty Parallels icon. It's poorly developed and embedded into the app itself rather than being an image file. So I've located this sed command that has some good feedback:

sudo sed -i.bak s/Parallels_Desktop_Overlay_128/Parallels_Desktop_Overlay_000/g /Applications/Parallels\ Desktop.app/Contents/MacOS/prl_client_app

It returns sed: RE error: illegal byte sequence

Can anyone explain what this means? What part of the command is the problem?

Maddie

Those coming from Google: try this answer first. I come to this thread more times than I'd like to admit. – Waligore

Answer 1

Try setting the LANG environment variable (LANG=C sed ...) or use one of the binary sed tools mentioned here: binary sed replacement

Why the error?

Without LANG=C sed assumes that files are encoded in whatever encoding is specified in LANG and the file (being binary) may contain bytes which are not valid characters in LANG's encoding (thus you could get 'illegal byte sequence').

Why does LANG=C work?

C just happens to treat all ASCII characters as themselves and non-ASCII characters as literals.

Panther

If bash is your shell, you can enter export LANG=C and try again. – Arrowhead

Great, but using LC_ALL=C sed ... is the more robust approach: if LC_ALL or LC_CTYPE are set (to something other than C), setting LANG will have no effect. (LC_ALL overrides all individually set categories, if any, whereas LANG only takes effect for those categories not explicitly set.) – Weekender

To me LANG=C did not work but LC_ALL=C did – Dorothadorothea

Answer 2

LANG=C alone didn't do the trick for me but adding LC_CTYPE=C as well solved it.

Fluent

An effective value of LC_CTYPE=C is sufficient to solve the problem: LC_CTYPE=C sed .... However, that won't work if LC_ALL is set (to something other than C), because that overrides all individual LC_* categories. Thus, the most robust approach is to use LC_ALL=C sed .... – Weekender

This worked for me on macOS Sierra – Orangy

Answer 3

In addition to LANG=C and LC_CTYPE=C, I had to do LC_ALL=C to get this to work.

LC_ALL overrides all individual LC_* categories. Thus, the most robust approach is to use LC_ALL=C sed ... - no need to also deal with the other variables.

Grapery

explain it a little bit more, please – Saundrasaunter

@rpax: Actually, an effective LC_CTYPE value of C is sufficient, so using LC_CTYPE=C sed ... (directly prepending to the offending command) is normally sufficient, unless LC_ALL - which overrides all individual LC_* categories - has been set. Thus, the most robust approach is to use LC_ALL=C sed ... - no need to also deal with the other variables. – Weekender

This should be the accepted answer. Hope you don't mind, I edited @mklement0's comment into the answer. – Waligore

@Qix: My contribution was added as a comment because I deemed it too invasive to be an edit: your edit amounts to putting words into @rjpeter2's mouth, and, in its current form, results in a somewhat self-contradictory answer. Do note that I've recommended the LC_ALL=C sed ... approach in comments on all answers on this page, including the currently accepted one. For the complete picture, I suggest consulting my own answer on the duplicate question. – Weekender

Answer 4

I managed to do it by running:

unset LANG

before the sed command.

Not sure what I've done or why it works but it did.

Mischance

I needed to do the above fix when following this tutorial: projectpoppycock.com/… – Dachi

This removes the error, but actually solves nothing for me. – Cornew

As for why it works: If your locale was initially set with [export] LANG=... (as opposed to setting LC_ALL or setting LC_* categories individually), then unsetting LANG makes all LC_* categories revert to "C"; ending up with a LC_CTYPE value of C is what solves the problem. Conversely, if LC_ALL or even LC_CTYPE specifically are set to something other than C, your approach won't work. – Weekender

@mklement0, I think your comments make a nice answer...! – Paschall

@Arjan: Thanks; I actually did create an answer - not to this, but a very similar question: https://mcmap.net/q/55072/-re-error-illegal-byte-sequence-on-mac-os-x – Weekender

(unset LANG; locale) <--- LC_CTYPE defaults to C – Windermere