Why are Google Colab shell commands not working?

Question

Steps to reproduce:

Open new Colab notebook on GPU

!ls #works
!pip install -q turicreate
import turicreate as tc
!ls #doesn't work

I get the following error:

---------------------------------------------------------------------------
NotImplementedError                       Traceback (most recent call last)
<ipython-input-22-16fdbe588ee8> in <module>()
----> 1 get_ipython().system('ls')
      2 # !nvcc --version

2 frames
/usr/local/lib/python3.6/dist-packages/google/colab/_system_commands.py in _run_command(cmd, clear_streamed_output)
    165   if locale_encoding != _ENCODING:
    166     raise NotImplementedError(
--> 167         'A UTF-8 locale is required. Got {}'.format(locale_encoding))
    168 
    169   parent_pty, child_pty = pty.openpty()

NotImplementedError: A UTF-8 locale is required. Got ANSI_X3.4-1968

Unfortunately of which makes little sense to me why this is occurring. Any leads? I will also post as a potential issue in the turicreate project.

EDIT:

It does look like it's overriding my locale as suggested in the comments. Before importing I can do:

import locale
locale.getdefaultlocale()
(en_US, UTF-8)

But after I get:

locale.getdefaultlocale()
(None, None)

Though I'm not sure how to reset the locale now that I've lost the use of shell commands?

Lopes

Colab is expecting to run in an environment with a UTF-8 locale; your's is not. You can set the locale from the terminal using the locale command. – Liebfraumilch

Relevant unix.stackexchange.com/questions/149111/… – Liebfraumilch

Answer 1

This is an issue with Turicreate. The relevant issue is opened here: https://github.com/apple/turicreate/issues/1862

The summary is: On startup turicreate sets the LC_ALL environment variable to C ().

The workaround for this:

import turicreate as tc
import os
del os.environ['LC_ALL']

Chiropody

Well, I'm the one who opened that issue ;) Though I should have also posted the answer here. Thanks. – Lopes

Answer 2

I got a different work around from a similar issue

First to check of the the current locale:

import locale
print(locale.getpreferredencoding())

The work around is to create a function that returns the required local i.e. UTF-8

import locale
def getpreferredencoding(do_setlocale = True):
    return "UTF-8"
locale.getpreferredencoding = getpreferredencoding

References:
OS module UTF Mode
Locale module

Pigeonhearted

Answer 3

got the same error, running following code in colab worked for me

import locale
locale.getpreferredencoding = lambda: "UTF-8"

Source: link.

Unbar

Answer 4

try this:

import locale
locale.getpreferredencoding = lambda: "UTF-8"
!pip install aspose-words

Psalm

Thank you for contributing to the Stack Overflow community. This may be a correct answer, but it’d be really useful to provide additional explanation of your code so developers can understand your reasoning. This is especially useful for new developers who aren’t as familiar with the syntax or struggling to understand the concepts. Would you kindly edit your answer to include additional details for the benefit of the community? – Envy

Answer 5

Try overrriding locale.getpreferredencoding:

Run the below code in a colab cell and then you can run your commands afterwards.

import locale
locale.getpreferredencoding = lambda: "UTF-8"

This should work!

Headwards