Rubyzip: Export zip file directly to S3 without writing tmpfile to disk?

Question

I have this code, which writes a zip file to disk, reads it back, uploads to s3, then deletes the file:

compressed_file = some_temp_path

Zip::ZipOutputStream.open(compressed_file) do |zos|
  some_file_list.each do |file|
    zos.put_next_entry(file.some_title)
    zos.print IO.read(file.path)
  end
end # Write zip file

s3 = Aws::S3.new(S3_KEY, S3_SECRET)
bucket = Aws::S3::Bucket.create(s3, S3_BUCKET)
bucket.put("#{BUCKET_PATH}/archive.zip", IO.read(compressed_file), {}, 'authenticated-read')

File.delete(compressed_file)

This code works already but what I want is to not create the zip file anymore, to save a few steps. I was wondering if there is a way to export the zipfile data directly to s3 without having to first create a tmpfile, read it back, then delete it?

Deforce

Answer 1

I think I just found the answer to my question.

It's Zip::ZipOutputStream.write_buffer. I'll check this out and update this answer when I get it working.

Update

It does work. My code is like this now:

compressed_filestream = Zip::ZipOutputStream.write_buffer do |zos|
  some_file_list.each do |file|
    zos.put_next_entry(file.some_title)
    zos.print IO.read(file.path)
  end
end # Outputs zipfile as StringIO

s3 = Aws::S3.new(S3_KEY, S3_SECRET)
bucket = Aws::S3::Bucket.create(s3, S3_BUCKET)

compressed_filestream.rewind
bucket.put("#{BUCKET_PATH}/archive.zip", compressed_filestream.read, {}, 'authenticated-read')

The write_buffer returns a StringIO and needs to rewind the stream first before reading it. Now I don't need to create and delete the tmpfile.

I'm just wondering now if write_buffer would be more memory extensive or heavier than open? Or is it the other way around?

Deforce

What is some_file_list ? – Sabo

I'm guessing its something like.. some_file_list = Zip::ZipFile.open(zipped_file) – Sabo

For those confused why you're getting errors like NameError: uninitialized constant Zip::ZipOutputStream, it's because nowadays the class is called Zip::OutputStream. – Caller