Sails.js checking stuff before uploading files to MongoDB with skipper (valid files, image resizing etc)

Question

I'm currently creating a file upload system in my application. My backend is Sails.js (10.4), which serves as an API for my separate front-end (Angular).

I've chosen to store the files I'm uploading to my MongoDB instance, and using sails' build in file upload module Skipper. I'm using the adapter skipper-gridfs (https://github.com/willhuang85/skipper-gridfs) to upload the files to mongo.

Now, it's not a problem to upload the files themselves: I'm using dropzone.js on my client, which sends the uploaded files to /api/v1/files/upload. The files will get uploaded.

To achieve this i'm using the following code in my FileController:

module.exports = {
    upload: function(req, res) {
        req.file('uploadfile').upload({
            // ...any other options here...
            adapter: require('skipper-gridfs'),
            uri: 'mongodb://localhost:27017/db_name.files'
        }, function(err, files) {
            if (err) {
                return res.serverError(err);
            }
            console.log('', files);
            return res.json({
                message: files.length + ' file(s) uploaded successfully!',
                files: files
            });
        });
    }
};

Now the problem: I want to do stuff with the files before they get uploaded. Specifically two things:

1. Check if the file is allowed: does the content-type header match the file types I want to allow? (jpeg, png, pdf etc. - just basic files).
2. If the file is an image, resize it to a few pre-defined sizes using imagemagick (or something similar).
3. Add file-specific information that will also be saved to the database: a reference to the user who has uploaded the file, and a reference to the model (i.e. article/comment) the file is part of.

I don't have a clue where to start or how to implement this kind of functionality. So any help would be greatly appreciated!

Empirical

Answer 1

Ok, after fiddling with this for a while I've managed to find a way that seems to work.

It could probably be better, but it does what I want it to do for now:

upload: function(req, res) {
    var upload = req.file('file')._files[0].stream,
        headers = upload.headers,
        byteCount = upload.byteCount,
        validated = true,
        errorMessages = [],
        fileParams = {},
        settings = {
            allowedTypes: ['image/jpeg', 'image/png'],
            maxBytes: 100 * 1024 * 1024
        };

    // Check file type
    if (_.indexOf(settings.allowedTypes, headers['content-type']) === -1) {
        validated = false;
        errorMessages.push('Wrong filetype (' + headers['content-type'] + ').');
    }
    // Check file size
    if (byteCount > settings.maxBytes) {
        validated = false;
        errorMessages.push('Filesize exceeded: ' + byteCount + '/' + settings.maxBytes + '.');
    }

    // Upload the file.
    if (validated) {
        sails.log.verbose(__filename + ':' + __line + ' [File validated: starting upload.]');

        // First upload the file
        req.file('file').upload({}, function(err, files) {
            if (err) {
                return res.serverError(err);
            }

            fileParams = {
                fileName: files[0].fd.split('/').pop().split('.').shift(),
                extension: files[0].fd.split('.').pop(),
                originalName: upload.filename,
                contentType: files[0].type,
                fileSize: files[0].size,
                uploadedBy: req.userID
            };

            // Create a File model.
            File.create(fileParams, function(err, newFile) {
                if (err) {
                    return res.serverError(err);
                }
                return res.json(200, {
                    message: files.length + ' file(s) uploaded successfully!',
                    file: newFile
                });
            });
        });
    } else {
        sails.log.verbose(__filename + ':' + __line + ' [File not uploaded: ', errorMessages.join(' - ') + ']');

        return res.json(400, {
            message: 'File not uploaded: ' + errorMessages.join(' - ')
        });
    }

},

Instead of using skipper-gridfs i've chosen to use local file storage, but the idea stays the same. Again, it's not as complete as it should be yet, but it's an easy way to validate simple stuff like filetype and size. If somebody has a better solution, please post it :)!

Empirical

Answer 2

You can specify a callback for the .upload() function. Example:

req.file('media').upload(function (error, files) {
  var file;

  // Make sure upload succeeded.
  if (error) {
    return res.serverError('upload_failed', error);
  }

  // files is an array of files with the properties you want, like files[0].size
}

You can call the adapter, with the file to upload from there, within the callback of .upload().

Berube

So if I understand this correctly, I should first use the default .upload function which will store the uploaded file(s) in the .tmp/uploads directory, and in the callback of the first upload function I should execute my custom stuff (like checking filetype etc), and then send it to Mongo using skipper-gridfs? Will I still be able to call .upload on the files in the first callback? – Empirical

I think so, yes. I don't see a validate() callback anywhere. – Berube

Maybe i'm doing something stupid, but when I try the following in the callback: files[0].upload({'adapter: require('skipper-gridfs'), uri: 'mongodb://localhost:27017/evolution_api_v1.files'}, function(err, files) { ... }); Console will die spitting out 'Object has no method upload'. Kinda like I expected. Also I feel like it's kinda weird that it's necessary to upload the file twice. It'd be nicer to intercept the file before it gets uploaded somehow.. Not sure if that's possible though :). – Empirical