JGroups, Terracotta & Hazelcast

Question

Trying to wrap my head around these 3 projects and they all seem to handle slightly different problems that arise when trying to cluster. But all the documentation for them is sort of written for developers that are already "in the know", and are difficult for a newbie like me to make good sense of.

• What are the specific problems each of them are trying to solve, and how do these problems differ from one another?
• How does clustering with each of them differ from clustering app servers (like JBoss's or GlassFish's built-in clustering capabilities)?
• Are the problems these frameworks solve different enough to warrant their use on the same project? Or are they competitors with each other and thus have different solutions to the same/similar problem?

Thanks in advance for any insight into these curious, yet elusive frameworks!

Novah

Answer 1

jgroups is more about task distribution and cluster management while hazelcast/terracotta are more distributed caches (data grids) - there is certainly overlap between them when you compare all the functionality - you need to figure out what functionality is more important and perhaps easier to implement.

hazelcast allows clustering through either tcp based addressing or multicasting. It supports maps, multimaps, lists, queues, topics - for disk based backups, you have to implement load/store interfaces.

With EhCache, you can use JGroups, JMS or RMI replication for caches.

In short, if you're looking for a distrubuted data cache/grid, hazelcast or ehcache would be the tools to look at - if you're looking for task distribution using a library and not concerned about existing data grid caches, JGroups would work for you.

Jehad

Thanks @ali (+1) - some quick follow-up though: what about my second question? How is clustering on JGroups/Hazelcast/Terracotta different than clustering my JBoss/GlassFish servers? Also, anything you can do to help me understand the difference/relationship between Terracotta and EhCache would be greatly appreciated! Thanks again! – Novah

terracotta boughtehcache and incorporated it into their product as the cache a few years ago. I am assuming you are looking at open source solutions (unless you wish to get the enterprise editions for these products). I have not spent too much time with JBoss/GlasshFish - I believe JBoss uses JGroups for memory replication/clustering (not sure but I think you could try to use JMS/RMI for that as well in the past but I might be wrong) - you can have disk based backups as well. Not sure about recent version of Glassfish. hope it helps. – Jehad

@Novah - AFAIK GlassFish uses Shoal for clustering. Shoal uses JXTA/JGroups under the hood. JBOSS uses a combination of Infinispan and JGroups, where Infinispan is the data grid solution and JGroups the clustering solution. Hazelcast and Terracotta are better comparable. Hazelcast can be used for both clustering and data grid. I cannot comment on its performance since haven't used it in production yet, but the framework is very simple to use. One more thing is, Hazelcast is designed to be a peer-to-peer solution whereas Terracotta is more of a master-slave type. – Collapse

Answer 2

It is possible to distinguish two categories of technologies: i. enabler (i.e. middleware API) and ii. end-to-end or ready-to-use solution (i.e. applicative API).

JGroups is an enabler technology as its core implements the Group Communication Primitives like Reliable Unicast, Multicast and Broadcast which are building blocks for more complex distributed protocols like Atomic Broadcast; it falls into the category of Group Communication Toolkits (GCTs).

Hazelcast as well as Terracotta are end-to-end service technologies as they provide a rich set of services for distributed applications; the fall into the category of In-Memory Data Grids (IMDGs), also known as distributed and in-memory caching solutions which are very well suited to compute data with low latencies.

In terms of capabilities:

• JGroups provides a set of primitives to enable group membership which is a key concept in any clustering scenario where a group of joining/leaving participants/nodes has to be managed in terms of lifecycle and roles; it allows to create a rich set of protocols based on a Protocol Kernel design by stacking micro protocols on the foundation APIs (as said, group membership) which rely on reliable distribution of messages both on TCP and UDP*. Out of the box, JGroups does not provide any composite service: such kind of services can be built on top of the basic provided capabilities.*

• Hazelcast provides a rich set of distributed data structures which can be fully replicated or sharded with an implicit replication factor; distributed List, Map, Queue and Lock are example of basic data structures implemented using the Java Collection interfaces, clearly the distribution and so replication implicitly needs Group Membership services which are provided by its Engine and in particular by the Cluster Manager with Cloud Discovery SPI module. Hazelcast can achieve Group Membership management via IP Multicast, IP Multicast with TCP and 3rd party Cloud Service (e.g. Zookeeper). Potentially, Hazelcast might use JGroups services for Node discovery and Cluster Management (aka Group Membership service).

• Terracotta, on the other hand, if associated to the popular Ehcache then it provides a Distributed Caching service which in turn is based on a set of basic Group Membership capabilities; in terms of implementation, Terracotta Ehcache may be based on top of JGroups services and provides a specific set of APIs specific of a Caching system and so less generic than Hazelcast.

Considering the relationship between the two types of technology, JGroups is effectively an enabling service (i.e. building block) for compound services (i.e. semantically rich APIs) like Hazelcast and Terracotta which provide end-to-end or ready-to-use services for 3rd party applications, managing all the reliable distribution aspects behind the scenes. Definitely, JGroups is a middleware and Hazelcast and Terracotta are applications which may embed their own middleware implementation for clustering services.

Epicycle