How can I check if a hazelcast cluster is alive from a java client?

Question

We use hazelcast in client-server mode. The hazelcast cluster contains 2 hazelcast nodes and we have about 25 clients connected to the cluster.

What I am lookin for now is a simple check that tries to figure out if the cluster is still alive. It should be a rather cheap operation because this check will occure on every client quite frequently (once every second I could imagine).

What is the best way to do so?

Ovotestis

Answer 1

The simplest way would be the register a LifecycleListener to the client HazelcastInstance:

    HazelcastInstance client = HazelcastClient.newHazelcastClient();
    client.getLifecycleService().addLifecycleListener(new LifecycleListener() {
        @Override
        public void stateChanged(LifecycleEvent event) {

        }
    })

The client uses a periodic heartbeat to detect if the cluster is still running.

Vulcanology

Sorry for the late response! I registered a listener and got almost immediate notification after I killed the cluster, so it basically worked. I only got 2 events though, CLIENT_CONNECTED and CLIENT_DISCONNECTED. Do I have to take care of the other events as well or are they not meant to be evaluated on client side? – Ovotestis

Answer 2

You can use the LifecycleService.isRunning() method as well:

HazelcastInstance hzInstance = HazelcastClient.newHazelcastClient();
hzInstance.getLifecycleService().isRunning()

Anabolism

finding that calling this on a client that is not connected (meaning there are no nodes in cluster and client repeatedly logging Unable to get alive cluster connection) this is returning true. – Vide

Answer 3

As isRunning() may be true even if cluster is down, I'd go for the following approach (a mixture of @konstantin-zyubin's answer and this). This doesn't need an event-listener, which is an advantage in my setup:

if (!hazelcastInstance.getLifecycleService().isRunning()) {
  return Health.down().build();
}

int parameterCount;
LocalTopicStats topicStats;
try {
  parameterCount = hazelcastInstance.getMap("parameters").size();
  topicStats = hazelcastInstance.getTopic("myTopic").getLocalTopicStats();
} catch (Exception e) {
  // instance may run but cluster is down:
  Health.Builder builder = Health.down();
  builder.withDetail("Error", e.getMessage());
  return builder.build();
}

Health.Builder builder = Health.up();
builder.withDetail("parameterCount", parameterCount);
builder.withDetail("receivedMsgs", topicStats.getReceiveOperationCount());
builder.withDetail("publishedMsgs", topicStats.getPublishOperationCount());
return builder.build();

Beatification

are there any rest endpoints that can be used to check? – Coop

To check what, @mike01010? My code is used in an endpoint to check my Hazelcast instance, but I guess this is not what you mean? – Beatification

im trying to figure out if there are endpoints to check the number of instances avaiable or if the queue is ready. i am assuming that if the queue is ready i can start submitting jobs and not have to wait for instances. – Coop

Answer 4

I have found a more reliable way to check hazelcast availability, because

client.getLifecycleService().isRunning()

when you use async reconnection mode is always return true, as was mentioned.

@Slf4j
public class DistributedCacheServiceImpl implements DistributedCacheService {

    private HazelcastInstance client;

    @Autowired
    protected ConfigLoader<ServersConfig> serversConfigLoader;

    @PostConstruct
    private void initHazelcastClient() {
        ClientConfig config = new ClientConfig();
        if (isCacheEnabled()) {
            ServersConfig.Hazelсast hazelcastConfig = getWidgetCacheSettings().getHazelcast();
            config.getGroupConfig().setName(hazelcastConfig.getName());
            config.getGroupConfig().setPassword(hazelcastConfig.getPassword());
            for (String address : hazelcastConfig.getAddresses()) {
                config.getNetworkConfig().addAddress(address);
            }

            config.getConnectionStrategyConfig()
                  .setAsyncStart(true)
                  .setReconnectMode(ClientConnectionStrategyConfig.ReconnectMode.ASYNC);

            config.getNetworkConfig()
                  .setConnectionAttemptLimit(0) // infinite (Integer.MAX_VALUE) attempts to reconnect
                  .setConnectionTimeout(5000);

            client = HazelcastClient.newHazelcastClient(config);
        }
    }

    @Override
    public boolean isCacheEnabled() {
        ServersConfig.WidgetCache widgetCache = getWidgetCacheSettings();
        return widgetCache != null && widgetCache.getEnabled();
    }

    @Override
    public boolean isCacheAlive() {
        boolean aliveResult = false;
        if (isCacheEnabled() && client != null) {
            try {
                IMap<Object, Object> defaultMap = client.getMap("default");
                if (defaultMap != null) {
                    defaultMap.size(); // will throw Hazelcast exception if cluster is down
                    aliveResult = true;
                }
            } catch (Exception e) {
                log.error("Connection to hazelcast cluster is lost. Reason : {}", e.getMessage());
            }
        }
        return aliveResult;
    }
}

Wilbur