Google API quickstart.py error KeyError: '_module'

Question

Using Gmail API. My client secret file is downloaded and working for Ruby. When I try the quickstart.py (python) version I get this error

File "quickstart.py", line 70, in <module>
    main()
  File "quickstart.py", line 55, in main
    credentials = get_credentials()
  File "quickstart.py", line 38, in get_credentials
    credentials = store.get()
  File "/Library/Python/2.7/site-packages/oauth2client/client.py", line 374, in get
    return self.locked_get()
  File "/Library/Python/2.7/site-packages/oauth2client/file.py", line 79, in locked_get
    credentials = Credentials.new_from_json(content)
  File "/Library/Python/2.7/site-packages/oauth2client/client.py", line 281, in new_from_json
    module = data['_module']
KeyError: '_module'

I have not changed the file at all, just added the client_secret.json to that working directory and also install the google-api-python-client. My python code came from here: https://developers.google.com/gmail/api/quickstart/python

Gypsophila

Any solution that worked for you? – Sacerdotal

I had the same issue with the KeyError: '_module' apparently it was due to storing not valid OAuth2 credentials (i.e. ServiceAccounts). Happened when working on separate branches for testing locally and merging with production. If that may help. – Backbone

Answer 1

Try replacing creds = store.get() with creds = None temporarily. If this works, you can refactor your code to always start with flow-based credentials instantiation. This worked for me. It seems Google samples are out of sync with their oauth2client.

Noland

Answer 2

oauth2client is trying to load credentials from a json file with an incorrect structure.

Maybe the Ruby client uses a different file format but I'd be surprised. Are you sure you didn't save client_secret.json as ~/.credentials/gmail-quickstart.json accidentally?

Regardless, removing ~/.credentials/gmail-quickstart.json and re-authenticating will generate a new credentials file with the correct structure.

Baldridge

Helpful explanation... +1 – Lemuroid

I've just used the exact structure that Google gave me and the file still won't be accepted - any more advice? – Claver

Reauthenticating, which gives you a 'client_secrets.json', will invalidate the data in 'credentials.json'; you have to delete it before running anything. If you have a 'settings.yaml' file you need to update the 'client_config:' data to correspond to your new 'client_id' and 'client_secret'. – Masuria

Answer 3

I am learning Python myself and had a similar problem, but with the Calendar API example. It turned out that it was a typo with regards to the SCOPE.

## Typo - Invalid definition
SCOPES = 'https://ww.googleapies.com/auth/calendar.readonly'
## Correct Value for SCOPE
SCOPES = 'https://www.googleapis.com/auth/calendar'

Also, Matt's answer help point me in the right direction. The gmail-quickstart.json is not the same thing as the client_secret.json. The client_secret.json allows you to make a request for an OAuth2 token. While the gmail-quickstart.json contains the issued token and meta-data associated with it. The gmail-quickstart.json isn't created until you successfully login.

One last thought, in order to log in successfully, the quickstart.py app launched an instance of my web-browser (Firefox) and went to the Google login screen. In order for Firefox to run properly, I had to set my DISPLAY variable properly first.

$ export DISPLAY=:0
$ xhost +
access control disabled, clients can connect from any host

Northcliffe

Answer 4

In this GitHub issue: error "KeyError: '_module'" when running gdrive_upload.py

sputnik-dev answared on 10 Jan 2016:

If someone have the same issue : auth_token.txt and client_secret.json are not the same! Don't link the client_secret.json from google API console. The file will be automatically created by the script.

Wrong way: gauth.SaveCredentialsFile("client_secret.json")

Right way: gauth.SaveCredentialsFile("<any random name>.json")

Extra:

PyDrive code that automate google drive api authetication. Use the browser just one time to authenticate and never more. It saves your credential data on mycreds.json :)

from pydrive.auth import GoogleAuth
from pydrive.drive import GoogleDrive

gauth = GoogleAuth()
gauth.LoadCredentialsFile("mycreds.json")

if gauth.credentials is None:
    gauth.LocalWebserverAuth()

elif gauth.access_token_expired:
    gauth.Refresh()

else:
    gauth.Authorize()

gauth.SaveCredentialsFile("mycreds.json")

Glycerin

Answer 5

I solved this by moving the client_secret.json to the same directory as the py file that is trying to read it (quickstart.py), mine was on the Desktop while i had saved the json to Documents. I saved the json to the Desktop too and boy, It flew!.

I dont know why it doesnt work when they are in different directories, defining a custom credential_path doesn't help.

Unmoor

Answer 6

I followed several recommendations listed here:

• moving the client_secret.json to the same directory as the py file that is trying to read it, as recommended by simic0de.

• Working with skirill idea, instead of completely eliminating the 'credentials=store.get()', I decided to handle the exception, so it would work with flow-based credentials instantiation.

The final code is:

try:
    credentials = store.get()
except:
    print('Working with flow-based credentials instantiation')

You can write your own code in the except line. This approach will allow the store.get() command to work when conditions are met.

Erst

Answer 7

The root cause for this error is that the current token.json (or storage.json) file you're using was written by an app using newer Python auth client libraries (meaning your code wrote the file with minimal content).

The oauth2client library is older and requires such a file (written by itself) include a _module entry in the JSON payload which doesn't exist. Sure, temporarily renaming creds = None works, but in reality, you can just blow away that token file and run your script again (and re-auth) without changing any of its code.

If you're confused by this and want to see examples like an old one using oauth2client and a new one using the replacement client libs where your code has to write the (abridged) token file that's incompatible with oauth2client, see both of the Drive samples I created at https://github.com/wescpy/gsuite-apis-intro/tree/master/python.

Chappie