Firebase cloud functions is very slow

We're working on an application that uses the new firebase cloud functions. What currently is happening is that a transaction is put in the queue node. And then the function removes that node and puts it in the correct node. This has been implemented because of the ability to work offline.

Our current problem is the speed of the function. The function itself takes about 400ms, so that's alright. But sometimes the functions take a very long time (around 8 seconds), while the entry was already added to the queue.

We suspect that the server takes time to boot up, because when we do the action once more after the first. It takes way less time.

Is there any way to fix this problem? Down here i added the code of our function. We suspect there's nothing wrong with it, but we added it just in case.

const functions = require('firebase-functions');
const admin = require('firebase-admin');
const database = admin.database();

exports.insertTransaction = functions.database
    .ref('/userPlacePromotionTransactionsQueue/{userKey}/{placeKey}/{promotionKey}/{transactionKey}')
    .onWrite(event => {
        if (event.data.val() == null) return null;

        // get keys
        const userKey = event.params.userKey;
        const placeKey = event.params.placeKey;
        const promotionKey = event.params.promotionKey;
        const transactionKey = event.params.transactionKey;

        // init update object
        const data = {};

        // get the transaction
        const transaction = event.data.val();

        // transfer transaction
        saveTransaction(data, transaction, userKey, placeKey, promotionKey, transactionKey);
        // remove from queue
        data[`/userPlacePromotionTransactionsQueue/${userKey}/${placeKey}/${promotionKey}/${transactionKey}`] = null;

        // fetch promotion
        database.ref(`promotions/${promotionKey}`).once('value', (snapshot) => {
            // Check if the promotion exists.
            if (!snapshot.exists()) {
                return null;
            }

            const promotion = snapshot.val();

            // fetch the current stamp count
            database.ref(`userPromotionStampCount/${userKey}/${promotionKey}`).once('value', (snapshot) => {
                let currentStampCount = 0;
                if (snapshot.exists()) currentStampCount = parseInt(snapshot.val());

                data[`userPromotionStampCount/${userKey}/${promotionKey}`] = currentStampCount + transaction.amount;

                // determines if there are new full cards
                const currentFullcards = Math.floor(currentStampCount > 0 ? currentStampCount / promotion.stamps : 0);
                const newStamps = currentStampCount + transaction.amount;
                const newFullcards = Math.floor(newStamps / promotion.stamps);

                if (newFullcards > currentFullcards) {
                    for (let i = 0; i < (newFullcards - currentFullcards); i++) {
                        const cardTransaction = {
                            action: "pending",
                            promotion_id: promotionKey,
                            user_id: userKey,
                            amount: 0,
                            type: "stamp",
                            date: transaction.date,
                            is_reversed: false
                        };

                        saveTransaction(data, cardTransaction, userKey, placeKey, promotionKey);

                        const completedPromotion = {
                            promotion_id: promotionKey,
                            user_id: userKey,
                            has_used: false,
                            date: admin.database.ServerValue.TIMESTAMP
                        };

                        const promotionPushKey = database
                            .ref()
                            .child(`userPlaceCompletedPromotions/${userKey}/${placeKey}`)
                            .push()
                            .key;

                        data[`userPlaceCompletedPromotions/${userKey}/${placeKey}/${promotionPushKey}`] = completedPromotion;
                        data[`userCompletedPromotions/${userKey}/${promotionPushKey}`] = completedPromotion;
                    }
                }

                return database.ref().update(data);
            }, (error) => {
                // Log to the console if an error happened.
                console.log('The read failed: ' + error.code);
                return null;
            });

        }, (error) => {
            // Log to the console if an error happened.
            console.log('The read failed: ' + error.code);
            return null;
        });
    });

function saveTransaction(data, transaction, userKey, placeKey, promotionKey, transactionKey) {
    if (!transactionKey) {
        transactionKey = database.ref('transactions').push().key;
    }

    data[`transactions/${transactionKey}`] = transaction;
    data[`placeTransactions/${placeKey}/${transactionKey}`] = transaction;
    data[`userPlacePromotionTransactions/${userKey}/${placeKey}/${promotionKey}/${transactionKey}`] = transaction;
}

Answers


firebaser here

It sounds like you're experiencing a so-called cold start of the function.

When your function hasn't been executed in some time, Cloud Functions puts it in a mode that uses fewer resources. Then when you hit the function again, it restores the environment from this mode. The time it takes to restore consists of a fixed cost (e.g. restore the container) and a part variable cost (e.g. if you use a lot of node modules, it may take longer).

We're continually monitoring the performance of these operations to ensure the best mix between developer experience and resource usage. So expect these times to improve over time.

The good news is that you should only experience this during development. Once your functions are being frequently triggered in production, chances are they'll hardly ever hit a cold start again.


Update - looks like a lot of these problems can be solved using the hidden variable process.env.FUNCTION_NAME as seen here: https://github.com/firebase/functions-samples/issues/170#issuecomment-323375462

Update with code - For example, if you have the following index file:

...
exports.doSomeThing = require('./doSomeThing');
exports.doSomeThingElse = require('./doSomeThingElse');
exports.doOtherStuff = require('./doOtherStuff');
// and more.......

Then all of your files will be loaded, and all of those files' requirements will also be loaded, resulting in a lot of overhead and polluting your global scope for all of your functions.

Instead separating your includes out as:

if (!process.env.FUNCTION_NAME || process.env.FUNCTION_NAME === 'doSomeThing') {
  exports.doSomeThing = require('./doSomeThing');
}
if (!process.env.FUNCTION_NAME || process.env.FUNCTION_NAME === 'doSomeThingElse') {
  exports. doSomeThingElse = require('./doSomeThingElse');
}
if (!process.env.FUNCTION_NAME || process.env.FUNCTION_NAME === 'doOtherStuff') {
  exports. doOtherStuff = require('./doOtherStuff');
}

This will only load the required file(s) when that function is specifically called; allowing you to keep your global scope much cleaner which should result in faster cold-boots.


This should allow for a much neater solution than what I've done below (though the explanation below still holds).


Original Answer

It looks like requiring files and general initialisation happening in the global scope is a huge cause of slow-down during cold-boot.

As a project gets more functions the global scope is polluted more and more making the problem worse - especially if you scope your functions into separate files (such as by using Object.assign(exports, require('./more-functions.js')); in your index.js.

I've managed to see huge gains in cold-boot performance by moving all my requires into an init method as below and then calling it as the first line inside any function definition for that file. Eg:

const functions = require('firebase-functions');
const admin = require('firebase-admin');
// Late initialisers for performance
let initialised = false;
let handlebars;
let fs;
let path;
let encrypt;

function init() {
  if (initialised) { return; }

  handlebars = require('handlebars');
  fs = require('fs');
  path = require('path');
  ({ encrypt } = require('../common'));
  // Maybe do some handlebars compilation here too

  initialised = true;
}

I've seen improvements from about 7-8s down to 2-3s when applying this technique to a project with ~30 functions across 8 files. This also seems to cause functions to need to be cold-booted less often (presumably due to lower memory usage?)

Unfortunately this still makes HTTP functions barely usable for user-facing production use.

Hoping the Firebase team have some plans in future to allow for proper scoping of functions so that only the relevant modules ever need to be loaded for each function.


I am facing similar issues with firestore cloud functions. The biggest is performance. Specially in case of early stage startups, when you can't afford your early customers to see "sluggish" apps. A simple documentation generation function for e.g gives this:

-- Function execution took 9522 ms, finished with status code: 200

Then: I had a straighforward terms and conditions page. With cloud functions the execution due to the cold start would take 10-15 seconds even at times. I then moved it to a node.js app, hosted on appengine container. The time has come down to 2-3 seconds.

I have been comparing many of the features of mongodb with firestore and sometimes I too wonder if during this early phase of my product I should also move to a different database. The biggest adv I had in firestore was the trigger functionality onCreate, onUpdate of document objects.

https://db-engines.com/en/system/Google+Cloud+Firestore%3BMongoDB

Basically if there are static portions of your site that can be offloaded to appengine environment, perhaps not a bad idea.


I have done these things as well, which improves performance once the functions are warmed up, but the cold start is killing me. One of the other issues I've encountered is with cors, because it takes two trips to the cloud functions to get the job done. I'm sure I can fix that, though.

When you have an app in its early (demo) phase when it is not used frequently, the performance is not going to be great. This is something that should be considered, as early adopters with early product need to look their best in front of potential customers/investors. We loved the technology so we migrated from older tried-and-true frameworks, but our app seems pretty sluggish at this point. I'm going to next try some warm-up strategies to make it look better


Need Your Help

Setting Oracle size of row fetches higher makes my app slower?

java performance oracle jdbc oracle11g

As detailed here and confirmed here, the default number of rows Oracle returns at time when querying for data over JDBC is 10. I am working on an app that to has to read and compare lots of data fr...

Children of XElement

c# xml linq-to-xml xelement

How do I get just the children of an XElement?