Concurrent drawing with paper.js

Question

I have an app that deals with drawing widget based upon paper.js. Several users can draw at the same time, and changes are broadcast to each other in real time. The problem is that I need to store the changes and show drawn image while document is loaded.

The natural solution is to store commands sent by clients in DB. But drawn images can consist of thousands of commands, and I can have tens of images. So when I'm opening a document getting a list of commands from server, drawing can take too much time.

Is there a better way of storing images and interaction between clients?

Note that I have a scaling feature, so storing raster is not an option.

UPDATE: If I'm storing an image (e.g. in a BLOB) it's not clear how to apply changes made in real time. Passing image each time is not the solution I want.

Madgemadhouse

Have you look at Firebase? They are quite good at doing this 3 way sync kind of thing. source, server and other clients at the same time... firebase.com/tutorial/#session/nvjq7b2kop2 – Gladdy

Answer 1

If you are going to save the drawing as an image you have a few possible solutions.

1. Save the item somewhere in a folder and save the directory path + file name in your database
2. Save images in the database as a blob. Blobs are really database intensive though.

There are some intresting articeles about blob's. Like this one from microsoft.

As expected from the common wisdom, objects smaller than 256K are best stored in a database while objects larger than 1M are best stored in the filesystem.

So performance wice it would be a better solution to save the image into a directory.

It's also possible to export a svgfile of an drawn image. (info) I don't know if this is any help to you, but this is my personal experience. And I agree with you that storing thousands of commands into a database isn't the best solution. So you might want to take a look at saving images somewhere, but then you would lose the ability to edit an image if you have that implemented.

Update:

If you don't want to save a blob the best solution would be to "render" the image each time an edit is made. So you could execute all commands when some one opens the drawing. And only apply the latest commands when an edit gets triggered.

There are several options to achieve this. Like Jimmy Chandra said, firebase would be a good solution. They also provide a tutorial with pretty much has everything you want to achieve. (drawing an image using x and ycoordinates real time) Maybe you have to take a look into that.

A bit more info about Firebase.

Firebase is a powerful API to store and sync data in realtime

This is exactly what you want to achieve I believe. You can try their full tutorial here.

An other option you might take into consideration is nodejs. I've seen people using nodejs for chat systems to send the data to all other users. If you can send data, I'm sure you can draw an image with it.

In the end it's up to you to choose which technology you want to use. So I think you can have to investigate a few solutions like I suggested and ask a different question if you run into trouble with integrating that technology.

Swear

that is not possible. If I store it a BLOB (whether it's raster or SVG), how can I store changes made in real time – Madgemadhouse

Have you considered saving the coordinates of the lines in the database? That would also mean a lot of lines in the database, but less as actions I think. I see you also have the option to save the image as a json format. That might help? Link – Swear

that is what I mean by The natural solution is to store commands sent by clients in DB. Actions (commands) or just coordinates do not matter. As the main problem is long time or repeating the list of actions (commands) – Madgemadhouse

I think you don't really have much options then. I think you have to "render" the image each time an edit is made. Like a full render when some one joins the drawing, and only render the latest changes when they occur. I think that would be your best approach. That way you don't have to execute all of the commands but only the last few. – Swear

Answer 2

Would saving data on client side as secondary storage be acceptable?

With html5 file api, it should be possible to keep a copy of all the save commands sent by clients in DB. So the next time it opens a document, the application will render the drawing from secondary storage while simultaneously query on server the latest updates, these updates will be appended once secondary storage finishes rendering.

With websockets, you can save the commands and it will broadcast the changes to other clients and hence update their drawing with the changes in real time.

Ferrate

Answer 3

2 years ago I developed something similar (in a team, closed code).

The first thing to note is that something like this is already solved by google docs. We read every blog post and article about how they did it and applied that to an multiuser SVG like simple editor. So here is what I remember:

The breakdown to simple commands is the right way to go. As you wrote, with them you can recreate your images very detailed. You can even answer exactly what an image looked at every requested timestamp.

Example commands might be

- change color
- change color again
- draw a line from .. to ..
- draw a line from .. to ..
- draw a line from .. to ..
- draw a line from .. to ..
- smooth that line
- change color
- change Stroke
- delete those lines and draw them again ;-)

Back to the goolge docs example, they save and broadcast every single character change (and more).

Yes this is a very clear and testable approach, but asking for performance requires another logic Layer. It is necessary to add a routine that simplifies these commands to an compressed version (see below)

The moment a user requests an image, your system looks for the latest compressed image and adds the single commands not included in your compress routine.

The compress routine:

Do a video recording of the creation of an dummy image (similar to what you expect from you users) in your favorite vector editor, you will note that about 60% to 80% percent of the commands are useless to the end result.

The targeted compress routine may have these three steps:

1. Most of the time you will see yourself in the recording repositioning points, handles or moving the objects, change the color again and again setting and undoing other parameters like stroke thickness. Even deleting an redoing of elements. The routine needs to optimize these commands to draw the end result directly. I'm sure, this will reduce your command list down to 10%-50% depending of the type of image you are looking for.

2. compressing commands. A Path in the example above is drawn in line segments, it needs to. You usually don't know on the first onMouseDown that it will end in a path with 100 segments and 200 Handles. But still like the other users to show the creation process. Add a command to your program that is able to draw complex things like a path. You won't need it in the user drawing process. This command can be used for faster recreation of complete shapes. A path with 100 points and 200 Handles can now be drawn in one command. Before that you had a list of at least 300 commands.

3. Reorder commands. Two path elements drawn by two users at the same time isn't easy to compress. A simple solution is to sort commands by element. The process might take 3 minutes to create each shape. An animation would require the real command order to see each shpae develop at the same time, but the end result doesn't care. Each shape can be much easier compressed if ordered by element.

sorry for the long post, hope it helps

Poleax

Answer 4

I'll give a wider answer with architectural recommendations so apologies in advance to perfectionists.

I'd use nginx and nodejs with websockets as my application layer, and use paperjs on my server side via npm https://www.npmjs.com/package/paper

For my persistence level I'd not only use a db cluster ( mysql/mongodb etc ) but also use redis as a commands cache.

Whenever commands are executed on canvas I'd persist it in redis and broadcast to all clients via socket.io, and finally on terminate/save/exit etc events caused by any user I'd persist it as an image in my db, or fs with a path reference in my db.

Hope this helps.

Patrizia