Storing data url in Mongo DB

Question

I am considering storing data-urls in my mongoDB instead of storing a reference to a file or using GridFS.

Data url:

data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAMcAAAEsCAYAAAB38aczAAAgAElEQV

All of the files I am storing are JPG or PNG, and are less than 1MB in size.

I am wondering if this is considered bad practice, and what the performance implications for both read and write operations, storing the data-urls 1) in a separate collection 2) as meta data in a collection.

I'm open to any other suggestions for small file storage.

Dyslexia

Answer 1

First, I wouldn't store base64 encoded data in a database that is perfectly capable of storing binary data. That's just waste of space. Store the image itself, not its base64 representation, i.e. not data : "VBORw0KGgoAAAANSUhEUgAAA...", but data : BinData("VBORw0KGgoAAAANSUhEUgAAA...") (the former is a string for MongoDB, the latter is binary data). Base64 increases the size by 33%.

Other than that, I think this is fine. The trade off is 1 request that grabs all the data vs. multiple requests. The downside of storing larger chunks of data is that all the data must be in RAM for a moment, but at 1MB that's probably a non-issue.

You should, however, make sure that you don't fetch the document in situations where you don't need the image. 1MB isn't too much, but for a read-heavy collection it's a disaster.

Velarium

Thank you. In my situation grabbing the image is rarely necessary, and several read-heavy queries are performed on the collection. I don't think it's a good idea to specify exclusions for all of my existing queries. So it looks like my options are to 1) just store the image on my fileserver, 2) store binary data in a separate collection, create refs, and use mongoose's populate to obtain the image when necessary. – Dyslexia

Sounds like the separate collection is best suited. I use the same concept in a pet project where the secondary collection is called... Images ;-) That works fine and for this usage pattern, it's better than GridFS – Velarium

Good to know that you have success with this technique. Thanks again mnemosyn for your knowledge. – Dyslexia

Answer 2

I just finished a solution for this. This solution works with Ajax, so you can use fetch calls in Javascript with this. The strange thing is that nowhere on the whole internet this solution could be found and thats why I put in on to help others who want to work with images with data uris :-)

Model:

        cmsImage: { data: Buffer, contentType: String }

Storing in MongoDB:

        let rawData = fs.readFileSync(`${root}public/uploads/` + file.originalname);
        let base64Data = Buffer.from(rawData).toString('base64');

        // upload this image
        let image = { 
            cmsImage: {
                data: base64Data,
                contentType: file.mimetype
            }    
        };

        // in this record in the database
        await this.model.findByIdAndUpdate(body.id, image);        

Retrieving from MongoDB and create image element dynamically:

       // decode image from database as image uri
       let imageArray = new Int8Array(image.data.data);
       let decodedImage = new TextDecoder().decode(imageArray);

       // image
       let cmsImage = document.createElement("img");
       cmsImage.src = "data:" + image.contentType + ";base64," + decodedImage;
       cmsImage.alt = "image";
       cmsContent.appendChild(cmsImage);

Multer - Use originalfile for upload to database.

      let storage = multer.diskStorage({
        destination: function (req, file, cb) {
          cb(null, './public/uploads')
        },
        filename: function (req, file, cb) {
          cb(null, file.originalname)
        }
      });

      this.upload = multer({ storage: storage });

upload to directory

      this.upload.single(uploadImage)

  

Mina