How to download entire folder from Firebase Storage?

Question

I want to download an entire folder from Firebase storage. I can download single files using DownloadURL as follows, but it does not work for folders.

var storageRef = firebase.storage().ref();

// Create a reference to the file we want to download
var starsRef = storageRef.child(path);

// Get the download URL
starsRef.getDownloadURL().then(function(url) {
  // Insert url into an <img> tag to "download"
  ImageUrl = url;

  console.log(ImageUrl);
}).catch(function(error) {
  switch (error.code) {
    case 'storage/object_not_found':
      // File doesn't exist
      break;

    case 'storage/unauthorized':
      // User doesn't have permission to access the object
      break;

    case 'storage/canceled':
      // User canceled the upload
      break;

    case 'storage/unknown':
      // Unknown error occurred, inspect the server response
      break;
  }
});

How to download entire folder from Firebase?

Aestivation

Question on what you expect the intended behavior to be: an array of all files, a zipped folder containing all files, or something else? – Elurd

@MikeMcDonald I want to download a zipped folder containing all files. – Aestivation

Here is an elegant solution that will solve all your problems, download all files at once as a single .zip file right on the client side: https://mcmap.net/q/357119/-how-to-take-a-list-of-downloadurls-and-donwload-a-zip-folder-of-all-the-files-from-firebase-storage (March 21, 2021) – Rosner

Answer 1

There is no API in Firebase Storage to download all files in a folder. You will have to download the files one by one, or create a zip file that contains all the files.

As Lahiru's answer shows it can be accomplished with gsutils, but that's a server-side operation - not something you'd run in your client-side application.

Related:

• How to get a list of all files in Cloud Storage in a Firebase app?

Garceau

I cannot find the way to get all the files and create a zip file from above related links. Please give me some tips to find a way to create a zip file that contains all the files. Thanks for your response. – Aestivation

Answer 2

You can use gsutil to download the whole storage bucket

gsutil -m cp -R gs://<bucket_name> .

Embodiment

easiest solution! – Nectar

This worked well. Installing gsutil was the most time-consuming part. – Encapsulate

Don't forget to enter the destination URL after the gsutil command like so: gsutil cp -r gs://<bucket_name>.appspot.com Downloads The destination URL must name a directory, bucket, or bucket subdirectory. – Essay

One thing that tripped me up for a bit was I forgot to add the . at the end in the answer above. That . is important :) – Bullins

refer to this for installing gsutil – Drawn

Thanks, this saved my life. I accidentally deleted files which were only there in my firebase storage. – Sit

But, this is system-level installation right? how to archive it via Node APIs? – Ludewig

Btw in windows you should run the shell in administrator to not get Access Denied – Shanteshantee

Appending folder name after the bucket url also works (if you want to download only one particular folder). – Bakery

Is it possible to run that from the backend Admin SDK? – Dimenhydrinate

Answer 3

Command gustil for Windows !!!

gsutil cp -r gs://<bucket_name>.appspot.com/OBJECT_NAME "D:\path"

Use Cloud tools for PowerShell

REF for install windows >> https://cloud.google.com/storage/docs/gsutil_install

Dazzle

gsutil is also available for Mac, terminal command is same as above (change dest. path!)... cloud.google.com/storage/docs/gsutil_install#mac – Petrarch

Answer 4

You can download the folder by creating a zip file of it.

Here is a sample function:

import JSZip from 'jszip';
import { saveAs } from 'file-saver';
import {
  getStorage,
  listAll,
  ref,
  getDownloadURL,
  getMetadata,
} from 'firebase/storage';
import { auth } from '../../Firebase';

export const downloadFolderAsZip = async () => {
  const jszip = new JSZip();
  const storage = getStorage();
  const folderRef = ref(
    storage,
    'images'
  );
  const folder = await listAll(folderRef);
  const promises = folder.items
    .map(async (item) => {
      const file = await getMetadata(item);
      const fileRef = ref(storage, item.fullPath);
      const fileBlob = await getDownloadURL(fileRef).then((url) => {
        return fetch(url).then((response) => response.blob());
      });
      jszip.file(file.name, fileBlob);
    })
    .reduce((acc, curr) => acc.then(() => curr), Promise.resolve());
  await promises;
  const blob = await jszip.generateAsync({ type: 'blob' });
  saveAs(blob, 'download.zip');
};

Terrorist

This solution is absolutely great, thank you very much! – Moorwort

glad it helped took me a couple hours to figure it out – Terrorist

Thanks! You don't probably don't need to call await getMetadata() for the name - item.name in place of file.name should work. – Upstate

Answer 5

For a recursive solution that includes subfolders in the zip file, see the following sample. You'll instantiate a jszip object, await promises from a function that zips files and traverses the directories, then save the zip. If the content is a file ("item"), it is zipped into the jszip object. If it is a folder ("prefix"), the function is called again with a new subpath, passing in the same jszip object. For further improvement, you may want to get contents with list and pagination if your contents are too many for listAll, since listAll limits retrievals.

import JSZip from 'jszip';
import { saveAs } from 'file-saver';
import {
  getStorage, ref, getBlob, listAll,
} from "firebase/storage";

const addFilesFromDirectoryToZip = async (directoryPath = "", zip) => {
  const storage = getStorage();
 
  const directoryContentsRef = ref(
    storage,
    directoryPath
  );
  const directoryContents = await listAll(directoryContentsRef);

  for (const file of directoryContents.items) {
    const fileRef = ref(storage, file.fullPath);
    const fileBlob = await getBlob(fileRef)
    zip.file(file.fullPath, fileBlob);
  }

  for (const folder of directoryContents.prefixes) {
    await addFilesFromDirectoryToZip(folder.fullPath, zip);
  };
};

export const downloadFolderAsZip = async (directoryPath = "") => {
  const zip = new JSZip();

  await addFilesFromDirectoryToZip(directoryPath, zip);

  const blob = await zip.generateAsync({ type: "blob" });
  const name = directoryPath.split('/').pop();
  saveAs(blob, name);
};

Upstate

Answer 6

Following the suggestions above, I made the .html script.

    <!DOCTYPE html>
<html>
<head>
  <title>Download Firebase Storage Folder</title>
</head>
<body>
  <h1>Download Firebase Storage Folder</h1>

  <button id="downloadButton">Download Folder</button>

  <script src="https://www.gstatic.com/firebasejs/8.6.1/firebase-app.js"></script>
  <script src="https://www.gstatic.com/firebasejs/8.6.1/firebase-storage.js"></script>
  <script src="https://cdnjs.cloudflare.com/ajax/libs/jszip/3.7.1/jszip.min.js"></script>
  <script src="https://cdnjs.cloudflare.com/ajax/libs/FileSaver.js/2.0.5/FileSaver.min.js"></script>

  <script>
    // Initialize Firebase with your configuration
    const firebaseConfig = {
            apiKey: 'apiKey....', //replace
            authDomain: 'YOUR_AUTH_DOMAIN',
            projectId: 'YOUR_PROJECT_ID',
            storageBucket: 'YOUR_STORAGE_BUCKET',
            messagingSenderId: 'YOUR_MESSAGING_SENDER_ID',
            appId: 'YOUR_APP_ID',
    };

    // Initialize Firebase
    const app = firebase.initializeApp(firebaseConfig);

    // Define the downloadFolderAsZip function here
    const downloadFolderAsZip = async () => {
      console.log("Starting folder download...");

      const jszip = new JSZip();
      const storage = firebase.storage(app);
      const folderName = '/[Your folder name]'; //Your folder name 

      try {
        const folderRef = storage.ref(folderName);
        console.log("Folder reference created:", folderRef);

        const downloadFiles = async (folderRef, path) => {
          const folder = await storage.ref(folderRef.fullPath).listAll();
          console.log("Folder:", path);
          console.log("Files in the folder:", folder.items);

          const promises = folder.items.map(async (item) => {
            console.log("Downloading file:", item.name);
            const file = await item.getMetadata();
            const fileRef = storage.ref(item.fullPath);
            const fileBlob = await fileRef.getDownloadURL().then((url) => {
              return fetch(url).then((response) => {
                if (!response.ok) {
                  console.error("Failed to fetch file:", item.name);
                  throw new Error('Failed to fetch file');
                }
                return response.blob();
              });
            });
            jszip.folder(path).file(item.name, fileBlob);
          });

          await Promise.all(promises);

          for (const subfolder of folder.prefixes) {
            await downloadFiles(subfolder, `${path}${subfolder.name}/`);
          }
        };

        await downloadFiles(folderRef, '');

        console.log("Number of files added to the ZIP:", jszip.files.length);

        if (jszip.files.length === 0) {
          console.log("ZIP is empty.");
          return;
        }

        const blob = await jszip.generateAsync({ type: 'blob' });
        saveAs(blob, 'download.zip');
        console.log("Download completed.");
      } catch (error) {
        console.error('Error:', error);
        alert('An error occurred while downloading the folder.');
      }
    };

    // Add an event listener to the "Download Folder" button
    const downloadButton = document.getElementById('downloadButton');
    downloadButton.addEventListener('click', () => {
      downloadFolderAsZip();
    });
  </script>
</body>
</html>

1. Fill the credentials
2. Fill the folder name
3. Save
4. Open the .html file with the browser
5. Be happy

Theorist

This give error in folder name, can you help me to solve error – Adventurous

I have a feeling, that you didn't change all placeholders. Also check Firebase permissions (Firebase Database => Rules) – Theorist

Answer 7

The gcloud CLI is now recommended over gsutil. Use the following command to download an entire bucket:

gcloud storage cp "bucket path" . -r

Docs: https://cloud.google.com/sdk/gcloud/reference/storage/cp

Lorelle

Answer 8

If you are working with Firebase Storage, it's recommended to use the gcloud storage commands (instead of gsutil)

Important: gsutil is not the recommended CLI for Cloud Storage. Use gcloud storage commands in the Google Cloud CLI instead. ref

Here's a page with some useful examples on how to use the CLI

In order to copy all files in a folder to a different folder, we can use the [--recursive, -R, -r] flag with the cp command:

gcloud storage cp --recursive fromDir toDir

For example:

gcloud storage cp --recursive gs://my-bucket-name.appspot.com/myFromFolder gs://my-bucket-name.appspot.com/myToFolder

Pleuropneumonia