Tmpfile error with django-import-export on Heroku

Question

I am using django-import-export to process CSV files uploaded to my Django admin site.

When I run Django on my local machine, all works fine.

When I deploy my application to Heroku I start getting errors related to tmpfile access:

Feb 24 14:35:12 test-staging app/web.3:  ERROR 2017-02-24 22:35:12,143 base 14 139687073408832 Internal Server Error: 
....
Feb 24 14:35:12 test-staging app/web.3:    File "/app/.heroku/python/lib/python2.7/site-packages/import_export/admin.py", line 163, in process_import 
Feb 24 14:35:12 test-staging app/web.3:      data = tmp_storage.read(input_format.get_read_mode()) 
Feb 24 14:35:12 test-staging app/web.3:    File "/app/.heroku/python/lib/python2.7/site-packages/import_export/tmp_storages.py", line 42, in read 
Feb 24 14:35:12 test-staging app/web.3:      with self.open(mode=mode) as file: 
Feb 24 14:35:12 test-staging app/web.3:    File "/app/.heroku/python/lib/python2.7/site-packages/import_export/tmp_storages.py", line 31, in open 
Feb 24 14:35:12 test-staging app/web.3:      return open(self.get_full_path(), mode) 
Feb 24 14:35:12 test-staging app/web.3:  IOError: [Errno 2] No such file or directory: u'/tmp/tmpvCUtrP' 

I've read up on what I can about Heroku ephemeral storage, it seems like this should work. I've verified I can create, view, and modify files in /tmp on a heroku dyno with my code via heroku run.

django-import-export has a module that allows you to overload the tempfile creation mechanism - but I'm not even sure what's wrong here (or, rather, why /tmp/tmpvCUtrP isn't getting created or isn't visible).

Hintz

Answer 1

django-import-export splits the import process across a few requests to the server.

Heroku can use multiple dynos, which do not share a common filesystem in /tmp.

By chance an initial request can got do dyno A and create the tmpfile in /tmp/tmpfilename, and then a later request can go to dyno B, and django-import-export requires /tmp/tmpfilename to be present - but it is not.

To get around this I switched to django-import-export's CacheStorage temporary storage method:

from import_export.tmp_storages import CacheStorage

class CustomAdminForm(ImportExportModelAdmin):
    tmp_storage_class = CacheStorage

Hintz

Answer 2

The root-cause-analysis in the answer by deadcode is correct. In environments where cache is not correctly configured, but the default_storage is (e.g. to S3), you can alternatively use the following:

from import_export.tmp_storages import MediaStorage

@admin.register(MyModel)
class MyModelAdmin(ImportExportMixin, admin.ModelAdmin):
    tmp_storage_class = MediaStorage

Adest

Yep, this works. By the way if anyone is looking for a good guide on how to enable S3 storage for your django app, this is a top-notch tutorial – Boyla