Cloud Functions: How to copy Firestore Collection to a new document?

Question

I'd like to make a copy of a collection in Firestore upon an event using Cloud Functions

I already have this code that iterates over the collection and copies each document

const firestore = admin.firestore()
firestore.collection("products").get().then(query => {
  query.forEach(function(doc) {
    var promise = firestore.collection(uid).doc(doc.data().barcode).set(doc.data());
  });
});

is there a shorter version? to just copy the whole collection at once?

Twaddle

Answer 1

Currently, no. Looping through each document using Cloud Functions and then setting a new document to a different collection with the specified data is the only way to do this. Perhaps this would make a good feature request.

How many documents are we talking about? For something like 10,000 it should only take a few minutes, tops.

Mosher

How did you calculate that? I only have 3000 docs but, I needed it to be instant. – Twaddle

Keep in mind that Firestore keeps each document in multiple data centers. There is no way to instantly copy thousands of documents over distributed data centers. The laws of physics are against us there. – Lorenzetti

some people might see that as a challenge :) but seriously speaking, this probably incurs a lot of cost for something that should be a cheap operation. could absolutely be a good feature request. – Inveigh

Or we could just use use the Cloud Firestore managed export and import service for this on specific collection(s)? - cloud.google.com/firestore/docs/manage-data/export-import – Shepard

Answer 2

I wrote a small nodejs snippet for this.

const firebaseAdmin = require('firebase-admin');
const serviceAccount = '../../firebase-service-account-key.json';
const firebaseUrl = 'https://my-app.firebaseio.com';

firebaseAdmin.initializeApp({
    credential: firebaseAdmin.credential.cert(require(serviceAccount)),
    databaseURL: firebaseUrl
});
const firestore = firebaseAdmin.firestore();

async function copyCollection(srcCollectionName, destCollectionName) {
    const documents = await firestore.collection(srcCollectionName).get();
    let writeBatch = firebaseAdmin.firestore().batch();
    const destCollection = firestore.collection(destCollectionName);
    let i = 0;
    for (const doc of documents.docs) {
        writeBatch.set(destCollection.doc(doc.id), doc.data());
        i++;
        if (i > 400) {  // write batch only allows maximum 500 writes per batch
            i = 0;
            console.log('Intermediate committing of batch operation');
            await writeBatch.commit();
            writeBatch = firebaseAdmin.firestore().batch();
        }
    }
    if (i > 0) {
        console.log('Firebase batch operation completed. Doing final committing of batch operation.');
        await writeBatch.commit();
    } else {
        console.log('Firebase batch operation completed.');
    }
}

copyCollection('customers', 'customers_backup').then(() => console.log('copy complete')).catch(error => console.log('copy failed. ' + error));

Muggy

Thanks this is really helpful. Will this copy all subcollection data as well? – Salas

Answer 3

This is the method i use to copy data to another collection, I used it to shift data (like sells or something) from an active collection to a 'sells feed' or 'sells history' collection.

At the top i reference the documents, at the bottom is the quite compact code. You can simply add a for loop on top for more than 1 operation.

Hope it helps somebody :)

DocumentReference copyFrom = FirebaseFirestore.instance.collection('curSells').doc('0001');
DocumentReference copyTo = FirebaseFirestore.instance.collection('sellFeed').doc('0001');

copyFrom.get().then((value) => {
  copyTo.set(value.data())
});

Greggrega

Doesn't take into consideration sub collections though – Uppity

Answer 4

There is no fast way at the moment. I recommend you rewrite your code like this though:

import { firestore }  from "firebase-admin";
async function copyCollection() {
    const products = await firestore().collection("products").get();
    products.forEach(async (doc)=> {
        await firestore().collection(uid).doc(doc.get('barcode')).set(doc.data());
    })
}

Margerymarget