I've been using django-import-export for a while now to provide CSV/XLS export functionality of data for users but as datasets get larger I'm encountering server timeouts.
I understand it's possible to use StreamingHttpResponse but from what I've seen this is done by writing your own CSV writer functions to output your data which, at least initially, doesn't strike me as something I can do with my current approach.
Is it possible to stream a response when you're using a django-import-export model resource to generate your file?
This is the kind of code I've implemented at the moment, with a standard HttpResponse;
class ExportConsolePlacesView(ClientPlacesView, View):
"""
Export view for all places, either incomplete or complete.
"""
model = Place
http_method_names = ['get', ]
def get(self, request, *args, **kwargs):
self.object_list = self.get_queryset()
console_name = self.console.name.replace(' ', '_')
if kwargs['query'] == u'complete':
dataset = PlaceResource().export(
Place.objects.complete_entrants_for_console(self.console)
)
filename = '{}_complete_entrants'.format(console_name).lower()
elif kwargs['query'] == u'incomplete':
dataset = PlaceResource().export(
Place.objects.incomplete_entrants_for_console(self.console)
)
filename = '{}_incomplete_entrants'.format(console_name).lower()
export_type = kwargs['format']
_dataset_methods = {
'csv': dataset.csv,
'xls': dataset.xls
}
response = HttpResponse(
_dataset_methods[export_type], content_type=export_type
)
response[
'Content-Disposition'] = 'attachment; filename={fn}.{ext}'.format(
fn=filename,
ext=export_type
)
return response