Redis docker not available on localhost of docker-compose

Question

I've currently ran into a problem i'm trying to solve for more than a week and i'm getting nowhere. I hope you can point me into the right direction.

Initial Situation

Description

The project i am building is a NestJS App which connects to some APIs. Internally, it uses bullmq as a message queue, which itself uses ioredis to connect to a redis database. I've connected my self-written server component as well as the redis (which uses docker) via docker-compose up with the following configuration:

version: '3'

services:
  server:
    image: myserver:1.4.0
    container_name: myserver
    depends_on:
      - db
    ports:
      - 3001:3000
    environment:
      - REDIS_HOST=redis
  db:
    image: redis:6.0.8
    container_name: redis
    ports:
      - 6379:6379

Versions

Workstation

• Docker version 19.03.13, build 4484c46d9d
• docker-compose version 1.27.4, build 40524192

Server Component

• bullmq 1.9.0
• ioredis 4.17.3

redis docker

• 6.0.8

Problem

The problem of my server-component is, that it tries to connect to the redis instance under the given REDIS_HOST at port 6379 using the following code:

readonly connection = new Redis(
    +(process.env.REDIS_PORT ?? this.configService.get('redis_port')),
    process.env.REDIS_HOST ?? this.configService.get('redis_host'),
  );

but throws the following error:

[ioredis] Unhandled error event: Error: connect ECONNREFUSED 127.0.0.1:6379

I expected it to just see the redis instance at the exposed port.

So, it doesn't see the redis instance at 127.0.0.1: but shouldn't it use the given ip?

Thing i checked

The server code is correct, the REDIS_HOST is correctly submitted and called in the ioredis call. So further digging inside ioredis i found this issue . So, it should be available given all the hints as locally on my workstation, i'm using 0.0.0.0:6379 to connect and it works just fine.

Docker compose does create a network bridge automatically, and using netcat i checked, port 6379 on the ip of the redis docker (as well as the aliases redis & db), the redis instance is available from the server dockers console.

I then explicitely set the subnet using the network configuration of docker-compose as well as giving the containers static ips, but as i already described: the ip is correctly resolved.

I found the following issue on the docker github issue 204. I think this is exactly the problem i am facing here, but how does one solve it?

tl;dr ioredis tries to connect to the correctly resolved ip of the redis instance, but fails, as the instance is not available on the local ip of the server component.

What my current state is

I sob uncontrollably.

I currently am out of ideas how to get the "myserver"-container to connect to the redis instance via ioredis. My point of view is, that the problem i am having has to be connected to the way docker on windows resolves ips to 127.0.0.1. .

• Is my point of view right?
• What other way can you suggest to try out?

Best regards & thanks in advance.

Edit (2020-11-27): After some digging and further investigating the suggestions of Jeffrey Mixon, I'm unfortuately not any closer to a solution. My last steps included:

• updating all dependencies (among others bullmq to v1.11, ioredis to 4.19.2). This did not change anything.
• I then found a relatively new post on the bullmq issue board of a similar problem and i switched from reusing the connection in a connection object like shown above to always creating a new connection, as its also explained in the bullmq docs. But this also did not help.

new Queue(name, {
        connection: {
          host: this.redisHost,
          port: this.redisPort,
        },
      })

• I then switched from using the 'Redis' Object from the IORedis library to the 'IORedis', but still: nothing changed in the habit of my application docker. Even though the Redis connection is correctly called with the redis host, it still tries to connect to 127.0.0.1:6379 as shown above.
• Lastly, the strange behavior, that if e.g i choose an unsolvable host url, the application docker correctly tries to connect to tthat unresolveable host). But as soon, as this host is available in the network of docker-compose, it uses 127.0.0.1.

Edit (2020-12-01): In the meantime, i checked on a clean linux machine if the problem could by happen only on docker-for-windows, but it does happen on linux as well.

I did not solve the problem itself, but i bypassed it for me by just putting everything inside one docker. As my application is more of a proof of concept, there is no big pain in doing so. I would leave the question open if there happens to be a solution in the future or more people having the same question.

For those wondering, my dockerfile including redis now stacks another layer on top of a redis image. I'm adding the parts prom the ng-cli-e2e image i used before. So in the beginning of my existing dockerfile i added:

FROM redis:6.0.9-buster

RUN apt update

RUN apt install nodejs -y

RUN apt install npm -y

In the end i created a small wrapper script which justs starts the redis server as well as my application. I'm also exposing two ports now, if i want to access everything from my machine.

EXPOSE 3000 6379

CMD ./docker-start-wrapper.sh

It's not the most beautiful solution, but it does work for the moment.

Congener

+1 this, whatever I try I end up with the following. Why is it so hard to connect with ioredis? Error: connect ECONNREFUSED 127.0.0.1:6379 at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1146:16) { errno: -111, code: 'ECONNREFUSED', syscall: 'connect', address: '127.0.0.1', port: 6379 } – Stereochemistry

check this document newbedev.com/… , db = redis.Redis(host='db', port=6379, decode_responses=True) – Nauseating

I'm not using flask, so i don't know what this information should give me. Your comment should be an answer, anyway. – Congener

Answer 1

The problem is that your application container is using localhost as the hostname for connecting to the redis container. It should be using the hostname redis in this case.

Consider the following demonstration:

version: '3.7'

services:
  server:
    image: busybox
    container_name: myserver
    entrypoint: /bin/sh
    # if redis is replaced by localhost or 127.0.0.1 below, this container will fail
    command: "-c \"sleep 2 && nc -v redis 6379\""
    depends_on:
      - db
    ports:
      - 3001:3000
    environment:
      - REDIS_HOST=redis
  db:
    image: busybox
    container_name: redis
    entrypoint: /bin/nc
    command: "-l -p 6379 -v"
    # is not necessary to publish these ports
    #ports:
    #  - 6379:6379

$ docker-compose -f scratch_2.yml up
Creating network "scratches_default" with the default driver
Creating redis ... done
Creating myserver ... done
Attaching to redis, myserver
redis     | listening on [::]:6379 ...
myserver  | redis (172.25.0.2:6379) open
redis     | connect to [::ffff:172.25.0.2]:6379 from [::ffff:172.25.0.3]:34821 ([::ffff:172.25.0.3]:34821)
myserver exited with code 0
redis exited with code 0

When you publish ports, they are for use outside the containers on the host. By attempting to connect your mysever container to 127.0.0.1, the container is simply attempting to connect to itself.

Lactation

Sorry for the delay. Thank you for your answer & especially the part about the "outside" ports. But unfortunatly, the answer does not gets me any further. As i stated in my question, i know that the connection should be to redis. And i just checked again, the host redis is correctly given to the application container. But still, as stated here github.com/luin/ioredis/issues/365 , ioredis would like to bind to an local port. how can i bind the port of the redis docker locally? – Congener

@Congener My point was that based on your error message ECONNREFUSED 127.0.0.1:6379, your application is not trying to connect to hostname redis despite the fact you may be trying to set it. Instead it is clearly trying to connect to 127.0.0.1 instead. I don't know anything about ioredis so it could just be a bug there. Have you tried the official redis node client redis? – Lactation

i justed used ioredis because the bullmq recommends so. But maybe i'll try to adapt to this. thanks for staying in touch! – Congener

@Congener bullmq needs the connectin options to any Queue but also Worker or QueueScheduler you instantiate. Can this be your problem? – Baldhead

@Baldhead is that so? docs.bullmq.io/guide/workers This could be indeed my problem. – Congener

Answer 2

The problem with docker-compose is that redis is not on localhost, but it is on its own net instead. By default, all the containers in a docker compose share the same default net, so your redis container should be available by all the other containers in that docker-compose with the host redis (or your container name, in your case db).

Another point to remark is that if you are using bullmq, not only the Queue options need a custom collection, but also any Worker or QueueScheduler that you use, so you shall pass the custom connection options also to them.

Baldhead

Well that's a good point. All my workers are instantiated with the queues, as in the bullmq docs. docs.bullmq.io/guide/workers does not show a way to share connection options with workers. As i stated in my question, i tried the host name redis as well as db with no luck. – Congener

When I faced this problem, adding "redis" (in my case) to all the queues and workers instances fixed the problem. If you have already done it and it still does not work, there is nothing that comes to my mind – Baldhead

Answer 3

Hopefully this helps, I just ran into this and turns out it was a code issue, setting up the right way to connect to redis via the client object was the cause of my issues, make sure your code matches the version of your redis module.

In my case, redis v4.6.6, the following worked:

// Create a redis client and ensure connection const redisClient = redis.createClient({ url: redis://${redisHost}:${redisPort}, });

Praxis