Websocket API to replace REST API? [closed]

Question

I have an application whose primary function works in real time, through websockets or long polling.

However, most of the site is written in a RESTful fashion, which is nice for application s and other clients in the future. However, I'm thinking about transitioning to a websocket API for all site functions, away from REST. That would make it easier for me to integrate real time features into all parts of the site. Would this make it more difficult to build applications or mobile clients?

I found that some people are already doing stuff like this: SocketStream

Discourteous

@Stegi long polling works well enough as a fallback, not super concerned about that. – Discourteous

Harry now after 7 years, how did it work to you? Wondering, since I want to move to that direction as well. @Discourteous – Pops

@DmitryKudryavtsev I ended up not doing so. Traditional method worked well for me and weren't much harder. – Discourteous

Answer 1

Not to say that the other answers here don't have merit, they make some good points. But I'm going to go against the general consensus and agree with you that moving to websockets for more than just realtime features is very appealing.

I am seriously considering moving my app from a RESTful architecture to more of an RPC style via websockets. This is not a "toy app", and I'm not talking about only realtime features, so I do have reservations. But I see many benefits in going this route and feel it could turn out to be an exceptional solution.

My plan is to use DNode, SocketIO, and Backbone. With these tools, my Backbone models and collections can be passed around from/to client and server by simply calling a functions RPC-style. No more managing REST endpoints, serializing/deserializing objects, and so forth. I haven't worked with socketstream yet, but it looks worth checking out.

I still have a long way to go before I can definitively say this is a good solution, and I'm sure it isn't the best solution for every application, but I'm convinced that this combination would be exceptionally powerful. I admit that there are some drawbacks, such as losing the ability to cache resources. But I have a feeling the advantages will outweigh them.

I'd be interested in following your progress exploring this type of solution. If you have any github experiments, please point me at them. I don't have any yet, but hope to soon.

Below is a list of to-read-later links that I've been collecting. I can't vouch that they are all worthwhile, as I've only skimmed many of them. But hopefully some will help.

---

Great tutorial on using Socket.IO with Express. It exposes express sessions to socket.io and discusses how to have different rooms for each authenticated user.

• http://www.danielbaulig.de/socket-ioexpress/

Tutorial on node.js/socket.io/backbone.js/express/connect/jade/redis with authentication, Joyent hosting, etc:

• http://fzysqr.com/2011/02/28/nodechat-js-using-node-js-backbone-js-socket-io-and-redis-to-make-a-real-time-chat-app/
• http://fzysqr.com/2011/03/27/nodechat-js-continued-authentication-profiles-ponies-and-a-meaner-socket-io/

Tutorial on using Pusher with Backbone.js (using Rails):

• http://blog.pusher.com/2011/6/21/backbone-js-now-realtime-with-pusher

Build application with backbone.js on the client and node.js with express, socket.io, dnode on the server.

• http://andyet.net/blog/2011/feb/15/re-using-backbonejs-models-on-the-server-with-node/
• http://addyosmani.com/blog/building-spas-jquerys-best-friends/
• http://fzysqr.com/2011/02/28/nodechat-js-using-node-js-backbone-js-socket-io-and-redis-to-make-a-real-time-chat-app/
• http://fzysqr.com/2011/03/27/nodechat-js-continued-authentication-profiles-ponies-and-a-meaner-socket-io/

Using Backbone with DNode:

• http://quickleft.com/blog/backbone-without-ajax-part-ii
• http://quickleft.com/blog/backbone-without-ajax-part-1
• http://sorensen.posterous.com/introducing-backbone-redis
• https://github.com/cowboyrushforth/minespotter
• http://amir.unoc.net/how-to-share-backbonejs-models-with-nodejs
• http://hackerne.ws/item?id=2222935
• http://substack.net/posts/24ab8c

Factfinding

I just answered a related question and included a few more thoughts: #4849142 – Factfinding

"I still have a long way to go before I can definitively say this is a good solution" - Just curiosity, was this really a good solution? :D – Emmuela

Please respond @Tauren. I am very interested in what you have to say now. – Porism

@Factfinding I am also curious as to how this worked out? – Antiparticle

It's 2020 and I'm curious too :-) – Masters

I am curious as well. – Purulent

Answer 2

HTTP REST and WebSockets are very different. HTTP is stateless, so the web server doesn't need to know anything, and you get caching in the web browser and in proxies. If you use WebSockets, your server is becoming stateful and you need to have a connection to the client on the server.

Request-Reply communication vs Push

Use WebSockets only if you need to PUSH data from the server to the client, that communication pattern is not included in HTTP (only by workarounds). PUSH is helpful if events created by other clients needs to be available to other connected clients e.g. in games where users should act on other clients behaviour. Or if your website is monitoring something, where the server pushes data to the client all the time e.g. stock markets (live).

If you don't need to PUSH data from the server, it's usually easier to use a stateless HTTP REST server. HTTP uses a simple Request-Reply communication pattern.

Mingle

We're very used to the one direction pattern because we've never had any alternatives before. But now as my app becomes more developed it's become more apparent to me that the more places in which push technology is used the more responsive, and the more engaging the app becomes. – Discourteous

My app shows a list of friends, and the amount of points they have for example. Why not update it in real time. If users can see their friends progressing then they might be more inclined to want to catch up. I have certain document models that while not exactly changed frequently, is changed enough that not having it update in real time might cause slight confusion. At some point enough of your site benefits from having push updates that you start looking at your code and half of it is about REST and the other half is about sockets and you say well, I want to unify this. – Discourteous

It is an option to use websockets only to push a notification/command to your webapp (like a getUpdate or refreshObjectWithId with params). This command could be analysed in your webapp (client) and followed by a rest request to get specific data instead of transporting the data itself through the websockets. – Rosin

There are a whole lot of reasons websockets can be easier than REST calls - not just for push. websocket.org/quantum.html – Varlet

WebSockets are amazing, and free the server to send the client data any time, not only in response to a client message. WebSockets implements a message-based protocol so clients can receive messages any time, and if they are waiting for a particular message, they can queue other messages for processing later, reorder queued messages, ignore pushed messages depending on app state, etc. I'll never write another REST-based application again. Flash makes it easy too, with open-source AS3-based WebSocket implementations and fallback to browser via ExternalInterface.(addCallback/call) methods. – Hyponitrite

Websockets being stateful is somewhat wrong. You need an app protocol to work with it. And you can easily build a stateless protocol on top. Just like http is a stateless protocol on top of stateful tcp. The only problem is that you have to do it manually. – Raynaraynah

Answer 3

The only problem I can using TCP (WebSockets) as your main web content delivery strategy is that there is very little reading material out there about how to design your website architecture and infrastructure using TCP.

So you can't learn from other people's mistakes and development is going to be slower. It's also not a "tried and tested" strategy.

Of course your also going to lose all the advantages of HTTP (Being stateless, and caching are the bigger advantages).

Remember that HTTP is an abstraction for TCP designed for serving web content.

And let's not forget that SEO and search engines don't do websockets. So you can forget about SEO.

Personally I would recommend against this as there's too much risk.

Don't use WS for serving websites, use it for serving web applications

However if you have a toy or a personal websites by all means go for it. Try it, be cutting-edge. For a business or company you cannot justify the risk of doing this.

Symphysis

Answer 4

I learned a little lesson (the hard way). I made a number crunching application that runs on Ubuntu AWS EC2 cloud services (uses powerful GPUs), and I wanted to make a front-end for it just to watch its progress in realtime. Due to the fact that it needed realtime data, it was obvious that I needed websockets to push the updates.

It started with a proof of concept, and worked great. But then when we wanted to make it available to the public, we had to add user session, so we needed login features. And no matter how you look at it, the websocket has to know which user it deals with, so we took the shortcut of using the websockets to authenticate the users. It seemed obvious, and it was convenient.

We actually had to spend quiet some time to make the connections reliable. We started out with some cheap websocket tutorials, but discovered that our implementation was not able to automatically reconnect when the connection was broken. That all improved when we switched to socket-io. Socket-io is a must !

Having said all that, to be honest, I think we missed out on some great socket-io features. Socket-io has a lot more to offer, and I am sure, if you take it in account in your initial design, you can get more out of it. In contrast, we just replaced the old websockets with the websocket functionality of socket-io, and that was it. (no rooms, no channels, ...) A redesign could have made everything more powerful. But we didn't have time for that. That's something to remember for our next project.

Next we started to store more and more data (user history, invoices, transactions, ...). We stored all of it in an AWS dynamodb database, and AGAIN, we used socket-io to communicate the CRUD operations from the front-end to the backend. I think we took a wrong turn there. It was a mistake.

• Because shortly after we found out that Amazon's cloud services (AWS) offer some great load-balancing/scaling tools for RESTful applications.
• We have the impression now that we need to write a lot of code to perform the handshakes of the CRUD operations.
• Recently we implemented Paypal integration. We managed to get it to work. But again, all tutorials are doing it with RESTful APIs. We had to rewrite/rethink their examples to implement them with websockets. We got it to work fairly fast though. But it does feel like we are going against the flow.

Having said all that, we are going live next week. We got there in time, everything works. And it's fast, but will it scale ?

Lennalennard

Just wondering as we are trying to make this decision ourselves, did it scale well with AWS? – Outrage

@Outrage apparently node can easily take 100's of socket-io connections on a cheap aws instance. We didn't notice any performance issues yet. One of the strange effects though, is that people who visit your website once, but then leave the website open in a tab, continue to use connections. (and that happens often on mobile phones). So, you need at least a kind of mechanism to kick out idle users. I haven't put effort yet in doing that though, as our performance does not suffer from it at all. - So, no scalling was necessary yet. – Lennalennard

In mean time, we're migrating Node.js code to NestJS, and while doing so we are rewriting all the authentication code. We will use a mix of REST and websocket APIs. We are also splitting up our application in smaller microservices, and that's really where NestJS makes our job a lot easier. – Lennalennard

4 months later, and now rewriting everything again, to use Cognito and Amplify of AWS to do the authentication for us. Amplify and Cognito have good support for social-accounts (google, facebook, ...) logins. – Lennalennard

Yet another update: In mean time GraphQL is gaining popularity, GraphQL also has a feature called "subscriptions". And AWS now has services like AppSync that support GraphQL subscriptions, and can even make them work with Lambdas in the future. - Considering that as well now. – Lennalennard

Answer 5

I would consider using both. Each technology has their merit and there is no one-size fits all solution.

The separation of work goes this way:

1. WebSockets would be the primary method of an application to communicate with the server where a session is required. This eliminates many hacks that are needed for the older browsers (the problem is support for the older browsers which will eliminate this)

2. RESTful API is used for GET calls that are not session oriented (i.e. not authentication needed) that benefit from browser caching. A good example of this would be reference data for drop downs used by a web application. However. can change a bit more often than...

3. HTML and Javascript. These comprise the UI of the webapp. These would generally benefit being placed on a CDN.

4. Web Services using WSDL are still the best way of enterprise level and cross-enterprise communication as it provides a well defined standard for message and data passing. Primarily you'd offload this to a Datapower device to proxy to your web service handler.

All of this happen on the HTTP protocol which gives use secure sockets via SSL already.

For the mobile application though, websockets cannot reconnect back to a disconnected session (How to reconnect to websocket after close connection) and managing that isn't trivial. So for mobile apps, I would still recommend REST API and polling.

Another thing to watch out for when using WebSockets vs REST is scalability. WebSocket sessions are still managed by the server. RESTful API when done properly are stateless (which mean there is no server state that needs to be managed), thus scalability can grow horizontally (which is cheaper) than vertically.

Authorization

Answer 6

Do I want updates from the server?

• Yes: Socket.io
• No: REST

The downsides to Socket.io are:

• Scalability: WebSockets require open connections and a much different Ops setup to web scale.
• Learnin: I don't have unlimited time for my learnin. Things have to get done!

I'll still use Socket.io in my project, but not for basic web forms that REST will do nicely.

Posthorse

Answer 7

WebSockets (or long polling) based transports mostly serve for (near) real-time communication between the server and client. Although there are numerous scenarios where these kinds of transports are required, such as chat or some kind of real-time feeds or other stuff, not all parts of some web application need to be necessarily connected bidirectionally with the server.

REST is resource based architecture which is well understood and offers it's own benefits over other architectures. WebSockets incline more to streams/feeds of data in real-time which would require you to create some kind of server based logic in order to prioritize or differentiate between resources and feeds (in case you don't want to use REST).

I assume that eventually there would be more WebSockets centric frameworks like socketstream in the future when this transport would be more widespread and better understood/documented in the form of data type/form agnostic delivery. However, I think, this doesn't mean that it would/should replace the REST just because it offers functionality which isn't necessarily required in numerous use cases and scenarios.

Sitdown

Answer 8

That's not a good idea. The standard isn't even finalized yet, support varies across browsers, etc. If you want to do this now you'll end up needing to fallback to flash or long polling, etc. In the future it probably still won't make a lot of sense, since the server has to support leaving connections open to every single user. Most web servers are designed instead to excel at quickly responding to requests and closing them as quickly as possibly. Heck even your operating system would have to be tuned to deal with a high number of simultaneous connections (each connection using up more ephemeral ports and memory). Stick to using REST for as much of the site as you can.

Recapitulate

Yes most webserves excel at HTTP. But node.js is not a web server, it's an io library. It can do TCP just fine. The question is basically saying can we design websites to use TCP instead of HTTP. – Symphysis

The same restrictions apply, you'll still run out of ephemeral ports/memory, it will still limit how many people you can serve simultaneously, and place unnecessary burden on the system. – Recapitulate

yes there's a limit but I don't think it's that big a deal if you don't create a new thread per connection. – Symphysis

I already have a socket for every user. global chat + newsfeed. – Discourteous

I guess in 2011 this was a great anwser. - So, I see where you are coming from. But in 2019, websockets have matured. – Lennalennard

Answer 9

I'd like to point out this blog post that is up to me, the best answer to this question.

In short, YES

The post contains all the best practices for such kind of API.

Atthia