Lettuce can't connect to Redis Cluster using SSL but can connect to same Redis server using SSL by treating it as a Standalone node

Question

I've an Azure Cache for Redis - Premium and Cluster enabled. I've been trying to connect to that Redis using spring-boot-starter-data-redis (spring boot version: 2.3.4.RELEASE, Java version: 11) and using lettuce client but Lettuce is throwing the following SSL exception when I am treating my Redis as a Redis Cluster but connects just fine when using it as a Standalone Redis server.

My pom.xml dependencies are:

<dependencies>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-data-redis</artifactId>
    </dependency>

    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-test</artifactId>
        <scope>test</scope>
        <exclusions>
            <exclusion>
                <groupId>org.junit.vintage</groupId>
                <artifactId>junit-vintage-engine</artifactId>
            </exclusion>
        </exclusions>
    </dependency>
</dependencies>

Java code:

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.CommandLineRunner;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.boot.autoconfigure.data.redis.RedisProperties;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.data.redis.connection.*;
import org.springframework.data.redis.connection.lettuce.LettuceClientConfiguration;
import org.springframework.data.redis.connection.lettuce.LettuceConnectionFactory;
import org.springframework.data.redis.core.StringRedisTemplate;


@Configuration
class LettuceConfig {

    @Bean
    StringRedisTemplate getStringRedisTemplate(final RedisProperties redisProperties) {
        return new StringRedisTemplate(getRedisConnectionFactory(redisProperties));
    }

    @Bean
    RedisConnectionFactory getRedisConnectionFactory(final RedisProperties redisProperties) {
    
        final RedisNode redisNode = RedisNode.newRedisNode()
                .listeningAt(redisProperties.getHost(), redisProperties.getPort())
                .build();

        // Connecting as a Redis Cluster
        final RedisClusterConfiguration redisClusterConfiguration = new RedisClusterConfiguration();
        redisClusterConfiguration.addClusterNode(redisNode);
        redisClusterConfiguration.setPassword(RedisPassword.of(redisProperties.getPassword()));

        // Connecting as a Standalone Redis server
        final RedisStandaloneConfiguration redisStandaloneConfiguration = new RedisStandaloneConfiguration();
        redisStandaloneConfiguration.setHostName(redisProperties.getHost());
        redisStandaloneConfiguration.setPort(redisProperties.getPort());
        redisStandaloneConfiguration.setPassword(RedisPassword.of(redisProperties.getPassword()));

        final LettuceClientConfiguration.LettuceClientConfigurationBuilder lettuceClientConfigurationBuilder =
                LettuceClientConfiguration.builder()
                .clientName(redisProperties.getClientName())
                .commandTimeout(redisProperties.getTimeout());

        if (redisProperties.isSsl()) {
            lettuceClientConfigurationBuilder.useSsl();
        }

        final LettuceClientConfiguration lettuceClientConfiguration = lettuceClientConfigurationBuilder.build();

        return new LettuceConnectionFactory(redisClusterConfiguration, lettuceClientConfiguration);
    }
}

@SpringBootApplication
public class LettuceClusterApplication implements CommandLineRunner {

    private final StringRedisTemplate stringRedisTemplate;

    @Autowired
    public LettuceClusterApplication(final StringRedisTemplate stringRedisTemplate) {
        this.stringRedisTemplate = stringRedisTemplate;
    }

    public static void main(String[] args) {
        SpringApplication.run(LettuceClusterApplication.class, args);
    }

    @Override
    public void run(String... args) throws Exception {
        System.out.println(stringRedisTemplate.hasKey("abc"));
    }
}

When using redisStandaloneConfiguration in new LettuceConnectionFactory(..., ...), the code works just fine, but if I use redisClusterConfiguration, the code fails with the following exception:

java.lang.IllegalStateException: Failed to execute CommandLineRunner
    at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:798) ~[spring-boot-2.3.4.RELEASE.jar:2.3.4.RELEASE]
    ...
Caused by: org.springframework.data.redis.RedisConnectionFailureException: Redis connection failed; nested exception is io.lettuce.core.RedisConnectionException: Unable to connect to [RedisURI [host='<redacted>.redis.cache.windows.net', port=6380]]
    at org.springframework.data.redis.connection.lettuce.LettuceExceptionConverter.convert(LettuceExceptionConverter.java:66) ~[spring-data-redis-2.3.4.RELEASE.jar:2.3.4.RELEASE]
    ...
Caused by: io.lettuce.core.RedisConnectionException: Unable to connect to [RedisURI [host='<redacted>.redis.cache.windows.net', port=6380]]
    at io.lettuce.core.RedisConnectionException.create(RedisConnectionException.java:78) ~[lettuce-core-5.3.4.RELEASE.jar:5.3.4.RELEASE]
    ...
Caused by: javax.net.ssl.SSLHandshakeException: No subject alternative names matching IP address <redacted> found
    ...
Caused by: java.security.cert.CertificateException: No subject alternative names matching IP address <redacted> found
    at java.base/sun.security.util.HostnameChecker.matchIP(HostnameChecker.java:165) ~[na:na]
    ...

My application.properties file:

spring.redis.host = <redacted>.redis.cache.windows.net
spring.redis.port = 6380
spring.redis.password = <redacted>
spring.redis.ssl = true
spring.redis.clientName = ${HOSTNAME}
spring.redis.timeout = 100000

Update: Found a similar issue in Github: https://github.com/lettuce-io/lettuce-core/issues/246 but it says that it should work with lettuce versions > 4.2 and my lettuce-core version (bundled under spring-boot-starter-data-redis) is 5.3.4.RELEASE. Also worth checking out is the documentation which states the same: https://lettuce.io/core/release/reference/#ssl

Lettuce supports SSL connections since version 3.1 on Redis Standalone connections and since version 4.2 on Redis Cluster

Raised GitHub issue as well: https://github.com/lettuce-io/lettuce-core/issues/1454

Judaism

Answer 1

If all of your nodes have the same IP address as the hostname (as is the case in Azure Cache for Redis, I think), then this is one way you could configure your client to map unresolved IP addresses back to the hostname listed in the certificate.

    @Bean
    ClientResources clientResources(RedisProperties redisProperties) throws UnknownHostException {
        var clientResourcesBuilder = DefaultClientResources.builder();
        var configuredHost = redisProperties.getHost();
        var inetAddresses = Arrays.asList(InetAddress.getAllByName(configuredHost));
        MappingSocketAddressResolver resolver = MappingSocketAddressResolver.create(
                DnsResolvers.UNRESOLVED,
                hostAndPort -> inetAddresses.stream()
                        .anyMatch(i -> i.getHostAddress().equals(hostAndPort.getHostText())) ?
                        HostAndPort.of(configuredHost, hostAndPort.getPort()) :
                        hostAndPort
        );
        clientResourcesBuilder.socketAddressResolver(resolver);
        return clientResourcesBuilder.build();
    }

As best I understand it, this is the most preferable of the solutions listed on the Github issue until Microsoft fixes things from their end.

Coquina

Answer 2

Unlike connecting in standalone mode, connecting to Azure redis in cluster mode is a two step process:

1. Connect to <hostname:6380>, authenticate, and fetch the cluster endpoint details
2. Connect to <ip address:port> that you got in the cluster endpoint details, authenticate again, and then send commands to the particular cluster shard your key is on

The reason you get No subject alternative names matching IP address <redacted> found is that Azure redis gives you an IP address + port number in the cluster endpoint details, and then Lettuce tries to validate your SSL connection against the IP address - instead of the hostname, but fails because its trying to verify the the SSL cert subject or SAN something.redis.cache.windows.net against the server endpoint that you are currently connecting to <ip address>:<port>.

You can get around this in most client libraries by configuring or overriding the SSL certificate validation to validate the server cert against your particular redis cache's hostname.

E.g. in .Net StackExchange.redis there's a config setting called 'sslhost' useful for this purpose.

Hopefully Lettuce has an equivalent.

Illomened

Note that an additional possible workaround would be to set .withVerifyPeer(false) but that is not good security. I think it would instead be ideal if Lettuce library could be improved to allow SSL verification to accept the server hostname of the cluster in this kind of scenario. – Illomened

Answer 3

We got the following error when just using spring.data.redis.ssl.enabled=true.

Cannot connect Redis Sentinel at rediss://username:***********************************@xxx.xxx.xxx.xxx:26379:
java.util.concurrent.CompletionException: javax.net.ssl.SSLHandshakeException: 
No subject alternative names matching IP address xxx.xxx.xxx.xxx found

After some trial and error we changed from lettuce (which is the default library to talk with redis in Spring Boot) to jedis and then it worked :)

We did that with the following code in pom.xml:

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-data-redis</artifactId>
    <exclusions>
        <exclusion>
            <groupId>io.lettuce</groupId>
            <artifactId>lettuce-core</artifactId>
        </exclusion>
    </exclusions>
</dependency>

<dependency>
    <groupId>redis.clients</groupId>
    <artifactId>jedis</artifactId>
</dependency>

We are using Spring Boot 3.1.5 and java 21.

Larisa