Zipkin not working in Docker - conncection refused

Question

Zipkin works well locally but not in docker container. All the microservices are registered well in the Eureka and they can communicate well. But the only problem is Zipkin. I am getting the following error:

org.springframework.web.client.ResourceAccessException: I/O error on POST request for "http://localhost:9411/api/v2/spans": Connect to http://localhost:9411 [localhost/127.0.0.1] failed: Connection refused

my docker-compose.yaml is as follows:

version: '3.8'

services:
  currency-exchange:
    image: samankt/springboot-udemy-currency-exchange:0.0.1-SNAPSHOT
    mem_limit: 512m
    ports:
      - '8000:8000'
    networks:
      - saman-network  
    environment:
      EUREKA.CLIENT.SERVICE-URL.DEFAULTZONE: http://naming-server:8761/eureka
      EUREKA.INSTANCE.PREFERIPADDRESS: true  
      SPRING.ZIPKIN.BASE-URL: http://zipkin-server:9411/
      RABBIT_URI: amqp://guest:guest@rabbitmq:5672
      SPRING_RABBITMQ_HOST: rabbitmq
      SPRING_ZIPKIN_SENDER_TYPE: rabbit
    depends_on:
      - naming-server
      - rabbitmq
      
  api-gateway:
    image: samankt/springboot-udemy-currency-api-gateway:0.0.1-snapshot
    mem_limit: 512m
    ports:
      - '8765:8765'
    networks:
      - saman-network  
    environment:
      EUREKA.CLIENT.SERVICE-URL.DEFAULTZONE: http://naming-server:8761/eureka
      EUREKA.INSTANCE.PREFERIPADDRESS: true  
      SPRING.ZIPKIN.BASE-URL: http://zipkin-server:9411/
      RABBIT_URI: amqp://guest:guest@rabbitmq:5672
      SPRING_RABBITMQ_HOST: rabbitmq
      SPRING_ZIPKIN_SENDER_TYPE: rabbit 
    depends_on:
      - naming-server
      - rabbitmq
      
  currency-converter:
    image: samankt/currency-conversion:0.0.1-SNAPSHOT
    mem_limit: 700m
    ports:
      - '8100:8100'
    networks:
      - saman-network  
    environment:
      EUREKA.CLIENT.SERVICE-URL.DEFAULTZONE: http://naming-server:8761/eureka
      EUREKA.INSTANCE.PREFERIPADDRESS: true  
      SPRING.ZIPKIN.BASE-URL: http://zipkin-server:9411/
      SPRING.ZIPKIN.DISCOVERYCLIENTENABLED: true 
      RABBIT_URI: amqp://guest:guest@rabbitmq:5672
      SPRING_RABBITMQ_HOST: rabbitmq
      SPRING_ZIPKIN_SENDER_TYPE: rabbit     
    depends_on:
      - naming-server
      - rabbitmq
      
  naming-server:
    image: samankt/naming-server:0.0.1-SNAPSHOT
    mem_limit: 512m
    ports:
      - '8761:8761'
    environment:
      SPRING.ZIPKIN.BASE-URL: http://zipkin-server:9411/    
    networks:
      - saman-network
      
  zipkin-server:
    image: openzipkin/zipkin:latest
    mem_limit: 400m
    ports:
      - '9411:9411'
    networks:
      - saman-network   
    environment:
      RABBIT_URI: amqp://guest:guest@rabbitmq:5672
    depends_on:
      - rabbitmq
    restart: always
      
      
  rabbitmq:
    image: rabbitmq:3.8.12-management
    ports:
      - '5672:5672'
      - '15672:15672'
    networks:
      - saman-network
    
networks: 
  saman-network:

Trafficator

Answer 1

SOLUTION 1

The error is telling you that the connection on http://localhost:9411/api/v2/spans is refused. When you run your application inside a docker container you need to send the request not with localhost, but through the docker network (in your case saman-network). To access the docker container which runs zipkin you need to send the request to http://zipkin-server:9411/api/v2/spans.

From Spring Boot 3 Sleuth is deprecated and you will need to use micrometer. To use tracing with zipkin you can add the following 4 dependencies to you pom.xml file

<dependency>
     <groupId>io.micrometer</groupId>
     <artifactId>micrometer-tracing-bridge-brave</artifactId>
</dependency>
<dependency>
    <groupId>io.micrometer</groupId>
    <artifactId>micrometer-observation-test</artifactId>
    <scope>test</scope>
</dependency>
<dependency>
    <groupId>io.zipkin.reporter2</groupId>
    <artifactId>zipkin-reporter-brave</artifactId>
</dependency>
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-actuator</artifactId>
</dependency>

Finally to change the Zipkin endpoint to send the tracing information you need to add the following to your application.yaml file.

management:
  tracing:
    sampling:
      probability: 1.0 # only for testing purpose, switch back to 0.1 for production code
  zipkin:
    tracing:
      endpoint: http://localhost:9411/api/v2/spans

It is good practice to use Spring profiles, therefore you copy your application.yaml file and name it application-docker.yaml. In this new file you can modify the Zipkin endpoint with the zipkin container name zipkin-server.

management:
  tracing:
    sampling:
      probability: 1.0 # only for testing purpose, switch back to 0.1 for production code
  zipkin:
    tracing:
      endpoint: http://zipkin-server:9411/api/v2/spans

You can activate the Spring profiles passing SPRING_PROFILES_ACTIVATE=docker in the environment.

---

SOLUTION 2

For you this solution is faster to implement but I didn't test it. You pass the Zipkin url as an environment value SPRING.ZIPKIN.BASE-URL inside the docker-compose.yml. This property was used with the Sleuth package. With Spring Boot 3 and micrometer try to pass instead: MANAGEMENT.ZIPKIN.TRACING.ENDPOINT: http://zipkin-server:9411/api/v2/spans

Plastid

If you're interested in more zipkin properties with spring boot 3 the class org.springframework.boot.actuate.autoconfigure.tracing.zipkin.ZipkinProperties defines them. – Plastid

Thanks @Plastid for your great solutions. In the first solution, you truly mentioned the problem in calling zipkin endpoint. But as you can see I changed the endpoint call by passing the right one into the SPRING.ZIPKIN.BASE-URL , but it seems that it does not work and localhost is called by default! I have not applied micormeter yet but going to test it as you recommended. I will inform you accordingly. The second solution points to the assignment of the environment variable of SPRING.ZIPKIN.BASE-UR that I did already but the problem persisted. Spring profile use is a great recommendation. – Trafficator

I was using zipkin-reporter-brave and micrometer-tracing-bridge-brave with Spring Boot 3.0.3. I can confirm that setting MANAGEMENT.ZIPKIN.TRACING.ENDPOINT solved my problem. Thanks :-) – Bentinck

Setting MANAGEMENT.ZIPKIN.TRACING.ENDPOINT is the right answer for me. – Canaletto

Answer 2

In my Spring Boot 3 project, I encountered the same kind of issue, and the following solution worked for me:

I added the following dependencies to my project:

<dependency>
    <groupId>io.micrometer</groupId>
    <artifactId>micrometer-observation</artifactId>
</dependency>
<dependency>
    <groupId>io.micrometer</groupId>
    <artifactId>micrometer-tracing-bridge-otel</artifactId>
</dependency>
<dependency>
    <groupId>io.opentelemetry</groupId>
    <artifactId>opentelemetry-exporter-zipkin</artifactId>
</dependency>

In my Docker Compose file, I set the environment variable as follows:

services:
  currency-exchange:
    image: XX
    ports:
      - XX
    networks:
      - XX
    depends_on:
      - XX
    environment:
      MANAGEMENT.ZIPKIN.TRACING.ENDPOINT:  http://zipkin-server:9411/api/v2/spans

Fingerling

Your answer could be improved with additional supporting information. Please edit to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers in the help center. – Cenac

Answer 3

The problem is solved when I downgraded to Spring 2.x from 3.0.

It seems that Zipkin does not work well with Spring 3.0 currently. The problem is for docker, not local running. I could not find enough resources to support this idea. But in my experience, this is the case. I tried Zipkin with "latest" tag with Spring 3.0.0 but the problem persists. But it does work well with Spring 2.x

Trafficator