What is the most efficient way to make a batch request to a Firebase DB based on an array of known keys?

Question

I need a solution that makes a Firebase DB API call for multiple items based on keys and returns the data (children) of those keys (in one response).

Since I don't need data to come real-time, some sort of standard REST call made once (rather than a Firebase DB listener), I think it would be ideal.

The app wouldn't have yet another listener and WebSocket connection open. However, I've looked through Firebase's API docs and it doesn't look like there is a way to do this.

Most of the answers I've seen always suggest making a composite key/index of some sort and filter accordingly using the composite key, but that only works for searching through a range. Or they suggest just nesting the data and not worrying about redundancy and disk space (and it's quicker), instead of retrieving associated data through foreign keys.

However, the problem is I am using Geofire and its query method only returns the keys of the items, not the items' data. All the docs and previous answers would suggest retrieving data either by the real-time SDK, which I've tried by using the once method or making a REST call for all items and filter with the orderBy, startAt, endAt params and filtering locally by the keys I need.

This could work, but the potential overhead of retrieving a bunch of items I don't need only to filter them out locally seems wasteful. The approach using the once listener seems wasteful too because it's a server roundtrip for each item key. This approach is kind of explained in this pretty good post, but according to this explanation it's still making a roundtrip for each item (even if it's asynchronously and through the same connection).

This poor soul asked a similar question, but didn't get many helpful replies (that really address the costs of making n number of server requests).

Could someone, once and for all explain the approaches on how this could be done and the pros/cons? Thanks.

Hazelwood

What a great question. I wish someone would answer it. – Undulatory

@Hazelwood How you dealt with this problem? I have post this similar question #40519076 – Gobble

Answer 1

Looks like you are looking for Cloud Functions. You can create a function called from http request and do every database read inside of it.

These function are executed in the cloud and their results are sent back to the caller. HTTP call is one way to trigger a Cloud Function but you can setup other methods (schedule, from the app with Firebase SDK, database trigger...). The data are not charged until they leave the server (so only in your request response or if you request a database of another region). Cloud Function billing is based on CPU used, number of invocations and running intances, more details on the quota section.

You will get something like :

const database = require('firebase-admin').database();
const functions = require('firebase-functions');

exports.getAllNodes = functions.https.onRequest((req, res) => {
  
  let children = [ ... ]; // get your node list from req
  let promises = [];

  for (const i in children) {
    promises.push(database.ref(children[i]).once('value'));
  }

  Promise.all(promises)
    .then(result => {
      res.status(200).send(result);
    })
    .catch(error => {
      res.status(503).send(error);
    });
});

That you will have to deploy with the firebase CLI.

Lilalilac

Answer 2

I need a solution that makes a Firebase DB API call for multiple items based on keys and returns the data (children) of those keys (in one response).

One solution might be to set up a separate server to make ALL the calls you need to your Firebase servers, aggregate them, and send it back as one response.

There exists tools that do this.

One of the more popular ones recently spec'd by the Facebook team is GraphQL. https://graphql.org/

Behind the scenes, you set up your graphql server to map your queries which would all make separate API calls to fetch the data you need to fit the query. Once all the API calls have been completed, graphql will then send it back as a response in the form of a JSON object.

Housewares

Answer 3

This is how you can do a one time call to a document in javascript, hope it helps

 // Get a reference to the database service
 let database = firebase.database();
 // one time call to a document
 database.ref("users").child("demo").get().then((snapshot) => {
     console.log("value of users->demo-> is", snapshot.node_.value_)
 });

Lidstone

This answer has very low quality. You are assuming API's are similar. Maybe they are not. Who knows... – Removable

I appologize i am relatively new here, still figuring out how to answer to questions. I have updated the answer please give me a Upvote if my edited answer is helpful – Lidstone

This could help you writing better answers. Tour, and How do I write a good answer?. – Removable