bulkloader not importing ndb.model

Question

I am still new to Python and GAE. I have an application on local server that is running just fine. I can add entity to my datastore, I can view my website, etc: everything is fine.

Now I am trying to use bulkloader to add entities to my datastore. I followed the tutorial at https://developers.google.com/appengine/docs/python/tools/uploadingdata. My loader is below:

from google.appengine.ext import ndb
from google.appengine.tools import bulkloader
import my_model

class ArticleLoader(bulkloader.Loader):
  def __init__(self):
    bulkloader.Loader.__init__(self, 'Article',
                               [('title', str),
                                ('author', str)
                                ])

loaders = [ArticleLoader]

I am getting the error:

No module named my_model

Does anyone have a fix for this?

Note: I am only using one directory. So my loader is in the same location as the other file that imports the my_model module.

Wincer

I don't have a bulkloader.yaml file. The tutorial mentions that one cannot be auto-generated for localhost. If I need one, does anyone have an example of what the file should look like? I am passing in a csv file. – Wincer

Answer 1

This can also happen if your PYTHONPATH is not properly set up. If you're on Linux, try running this before you run the Bulkloader:

export PYTHONPATH=$PYTHONPATH:.

This appends your current directory to your PYTHONPATH and should make your my_model module visible. Since my memory is terrible and I always forget to do it, I've ended up using a simple shell script that includes this at the beginning and then the bulkload command itself.

If you're on Windows, you should be able to modify your path by using sys.path.append. Haven't tested this, but you could try adding this to your script (note that this should work on Linux as well):

import sys
# ...
sys.path.append('.')

Regolith

Cool! Up vote! Now I am getting a new error along the same line: raise KindError('No implementation for kind \'%s\'' % kind) google.appengine.ext.db.KindError: No implementation for kind 'Article' – Wincer

So now it sees the module but not the class inside the module -- which is actually the only class inside the module. Every other file sees the class just fine -- except this loader. – Wincer

Do I need a bulkloader.yaml? Since I cannot auto-generate one, do you have an example I may use? I am passing in a csv file. – Wincer

@Wincer Ah, you're using ndb, didn't see that at first. You can actually see the cause of the error in your traceback - the db API is the one being called, not ndb. I remember somebody posting a similar question - I'll paste the link here if I can find it (I think it is actually a missing feature). – Regolith

@Wincer Here is the active feature request (code.google.com/p/appengine-ndb-experiment/issues/detail?id=224). This came from #13571966, which was answered by someone with much more credibility than I :) Sorry for the unsatisfying answer. – Regolith

Many thanks. At least now I know why. Since google is pushing for ndb. They should really consider adding a way to load csv files. – Wincer

@Wincer Sure thing. I'm sure it will get implemented at some point - loading from CSV with db is extremely useful. There may be some workarounds out there, though I don't know if they are Google-approved. In any case, good luck with everything. – Regolith

Answer 2

Your code should be located in a file named my_model.py. You are getting that error because there is no module named my_module. Might be worth a read of the Python module and package docs.

Mana

I don't like to down vote people who are kind enough to try to help. But you should really read the question better next time. – Wincer