Is it possible to generate and return a ZIP file with App Engine?

Question

I have a small project that would be perfect for Google App Engine. Implementing it hinges on the ability to generate a ZIP file and return it.

Due to the distributed nature of App Engine, from what I can tell, the ZIP file couldn't be created "in-memory" in the traditional sense. It would basically have to be generated and and sent in a single request/response cycle.

Does the Python zip module even exist in the App Engine environment?

Coleorhiza

Answer 1

zipfile is available at appengine and reworked example follows:

from contextlib import closing
from zipfile import ZipFile, ZIP_DEFLATED

from google.appengine.ext import webapp
from google.appengine.api import urlfetch

def addResource(zfile, url, fname):
    # get the contents      
    contents = urlfetch.fetch(url).content
    # write the contents to the zip file
    zfile.writestr(fname, contents)

class OutZipfile(webapp.RequestHandler):
    def get(self):
        # Set up headers for browser to correctly recognize ZIP file
        self.response.headers['Content-Type'] ='application/zip'
        self.response.headers['Content-Disposition'] = \
            'attachment; filename="outfile.zip"'    

        # compress files and emit them directly to HTTP response stream
        with closing(ZipFile(self.response.out, "w", ZIP_DEFLATED)) as outfile:
            # repeat this for every URL that should be added to the zipfile
            addResource(outfile, 
                'https://www.google.com/intl/en/policies/privacy/', 
                'privacy.html')
            addResource(outfile, 
                'https://www.google.com/intl/en/policies/terms/', 
                'terms.html')

Piccaninny

Keep in mind the response size limit on App Engine is 10MB so you cannot return a zip file larger than that. Perhaps using the new Files API (SDK 1.4.3) you could create the zip file, store it in the blobstore, then return the blob. – Weikert

This answer fails at 'buf=zipf.read(2048)'; there's no earlier reference to 'zipf'; use the answer below – Brodeur

@BryceCutt does Google Cloud storage change the picture at all? – Morpheme

@Morpheme you can write a zip file to Google Cloud Storage much like the blobstore using the Files API (developers.google.com/appengine/docs/python/googlestorage) however unless you need some other feature of Cloud Storage (like sharing between apps) I don't see the benefit of it over the blobstore. It's nice to have options though. – Weikert

If you are copy-pasting this and using other data (especially data direct from the datastore), make sure to encode the filename and the contents, for example contents.encode('utf-8') as there is no standard and it may lead to errors – Plumper

I keep getting: AttributeError: 'Response' object has no attribute 'tell' - Any ideas? – Elaterite

Webapp is not available in GAE anymore. According to the webapp2 official docs: "The response buffers all output in memory, then sends the final output when the handler exits. webapp2 does not support streaming data to the client." So, it seems that this code is not actually streaming the response as the answer suggests. – Ballman

Answer 2

import zipfile
import StringIO

text = u"ABCDEFGHIJKLMNOPQRSTUVWXYVabcdefghijklmnopqqstuvweyxáéöüï东 廣 広 广 國 国 国 界"

zipstream=StringIO.StringIO()
file = zipfile.ZipFile(file=zipstream,compression=zipfile.ZIP_DEFLATED,mode="w")
file.writestr("data.txt.zip",text.encode("utf-8"))
file.close()
zipstream.seek(0)
self.response.headers['Content-Type'] ='application/zip'
self.response.headers['Content-Disposition'] = 'attachment; filename="data.txt.zip"'
self.response.out.write(zipstream.getvalue())

Disengage

Answer 3

From What is Google App Engine:

You can upload other third-party libraries with your application, as long as they are implemented in pure Python and do not require any unsupported standard library modules.

So, even if it doesn't exist by default you can (potentially) include it yourself. (I say potentially because I don't know if the Python zip library requires any "unsupported standard library modules".

Shugart