Docker compose with .NET Core, SQL Server, Elasticsearch, and cerebro services

I'm trying to run a number of services using a docker-compose file. First of all let's say that Docker, version 20.10.3, is running on a Red Hat Enterprise Linux release 8.3.

This is the docker-compose file:

version: "3.8"
services:
  projmssql:
    image: "mcr.microsoft.com/mssql/server:2019-latest"
    container_name: proj-core-sqlserver
    environment:
        SA_PASSWORD: "mypassword"
        ACCEPT_EULA: "Y"
    ports:
      - "1401:1433"
    volumes:
      - type: bind
        source: ./data-mssql
        target: /var/opt/mssql/data
      - type : bind
        source: ./log-mssql
        target: /var/opt/mssql/log
    restart: always
  projelastic:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.10.1
    container_name: proj-core-elastic
    environment:
      - node.name=es01
      - cluster.name=proj-docker-cluster
      - discovery.type=single-node
      - bootstrap.memory_lock=false
      - "ES_JAVA_OPTS=-Xms1g -Xmx1g"
      - path.repo=/usr/share/elasticsearch/backup
      - path.logs=/usr/share/elasticsearch/logs
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - type: bind
        source: ./data-es01
        target: /usr/share/elasticsearch/data
      - type : bind
        source: ./_backup-es01
        target: /usr/share/elasticsearch/backup
      - type : bind
        source: ./logs-es01
        target: /usr/share/elasticsearch/logs
    ports:
      - 9220:9200
      - 9320:9300
  projcerebro:
    image: lmenezes/cerebro
    container_name: proj-core-cerebro
    ports:
      - "9020:9000"
    command:
      - -Dhosts.0.host=http://projelastic:9200
  projapi:
    image: proj/projcoreapp 
    depends_on:
       - projmssql
    container_name: proj-core-api
    ports:
      - "8080:80"
    restart: always
    #Specify Environment Variables for the Api Service
    environment: 
      - ASPNETCORE_ENVIRONMENT=Docker

The projApi service comes from my local image (proj/projcoreapp) built with the following Dockerfile:

FROM mcr.microsoft.com/dotnet/sdk:5.0 AS build-env
WORKDIR /app

COPY . ./
RUN dotnet publish proj.api.net -c Release -o out

FROM mcr.microsoft.com/dotnet/aspnet:5.0
WORKDIR /app
COPY --from=build-env /app/out .

ENTRYPOINT ["dotnet", "Proj.Api.dll"]

In my .net core app I have a dedicated appsettings.Docker.json file

{
  "Logging": {
    "LogLevel": {
      "Default": "Information",
      "Microsoft": "Warning",
      "Microsoft.Hosting.Lifetime": "Information"
    }
  },
  "AllowedHosts": "*",
  "Elasticsearch": {
    "Server": "http://projelastic:9200",
    "DebugMode": true,
    "UpdateMapping": false
  },
  "ConnectionStrings": {
    "DatabaseConnection": "Server=projmssql;Database=PROJ-NET;Persist Security Info=False;User ID=sa;Password=mypassword;MultipleActiveResultSets=True;TrustServerCertificate=False;Connection Timeout=30;"
  }
}

The .csproj of the API .net core project

<Project Sdk="Microsoft.NET.Sdk.Web">

  <PropertyGroup>
    <TargetFramework>net5.0</TargetFramework>
  </PropertyGroup>

  <PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|AnyCPU'">
    <DocumentationFile></DocumentationFile>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Microsoft.AspNetCore.Authentication" Version="2.2.0" />
    <PackageReference Include="Microsoft.AspNetCore.Authentication.AzureAD.UI" Version="5.0.5" />
    <PackageReference Include="Microsoft.AspNetCore.Mvc.Versioning" Version="5.0.0" />
    <PackageReference Include="Microsoft.Identity.Web" Version="1.9.1" />
    <PackageReference Include="Microsoft.VisualStudio.Azure.Containers.Tools.Targets" Version="1.10.14" />
    <PackageReference Include="Swashbuckle.AspNetCore" Version="5.6.3" />
  </ItemGroup>

  <ItemGroup>
    <ProjectReference Include="..\domain.net\Domain.csproj" />
  </ItemGroup>

</Project>

The .csproj of the Domain library

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFramework>net5.0</TargetFramework>
  </PropertyGroup>

  <ItemGroup>
    <Compile Remove="Entities\Organisation.cs" />
    <Compile Remove="Entities\OrganisationTeam.cs" />
    <Compile Remove="Entities\OrganisationTeamRole.cs" />
  </ItemGroup>

  <ItemGroup>
    <PackageReference Include="Elasticsearch.Net" Version="7.10.0" />
    <PackageReference Include="Microsoft.EntityFrameworkCore" Version="5.0.5" />
    <PackageReference Include="Microsoft.EntityFrameworkCore.SqlServer" Version="5.0.5" />
    <PackageReference Include="NEST" Version="7.10.0" />
    <PackageReference Include="Newtonsoft.Json" Version="13.0.1" />
  </ItemGroup>

  <ItemGroup>
    <Folder Include="Utilities\" />
    <Folder Include="ViewModels\" />
  </ItemGroup>

  <ItemGroup>
    <ProjectReference Include="..\filter.net\Filter.csproj" />
  </ItemGroup>

</Project>

The .csproj of the Filter library

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFramework>net5.0</TargetFramework>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Elasticsearch.Net" Version="7.10.0" />
    <PackageReference Include="GeoJSON.Net" Version="1.2.19" />
    <PackageReference Include="NEST" Version="7.10.0" />
    <PackageReference Include="Newtonsoft.Json" Version="13.0.1" />
  </ItemGroup>

  <ItemGroup>
    <Folder Include="Entities\" />
    <Folder Include="Helpers\" />
    <Folder Include="ViewModelMapper\" />
  </ItemGroup>

</Project>

In my .net core startup I have the following configuration

public void ConfigureServices(IServiceCollection services)
        {
            services.AddDbContext<ProjdbContext>(options =>
                options.UseSqlServer(Configuration.GetConnectionString("DatabaseConnection"))
            );
            services.AddScoped<IProjUserService, ProjUserService>();
            services.AddSingleton<IProjEntitiesService>(s =>
                new ProjEntitiesService
                (
                    Configuration.GetValue<string>("Elasticsearch:Server"),
                    Configuration.GetValue<bool>("Elasticsearch:DebugMode"),
                    Configuration.GetValue<bool>("Elasticsearch:UpdateMapping")
                ));

            services.AddControllers();
            services.AddSwaggerGen(c =>
            {

Now let's talk about the issue: once I run the docker-compose file, all the services run correctly and from the host I can interact with all of them. The API service is able to connect internally to the database but once I run some APIs to interact with the elasticsearch service I get the following error:

Elasticsearch.Net.UnexpectedElasticsearchClientException: The information requested is unavailable on the current platform.
 ---> System.PlatformNotSupportedException: The information requested is unavailable on the current platform.
   at System.Net.NetworkInformation.StringParsingHelpers.ParseActiveTcpConnectionsFromFiles(String tcp4ConnectionsFile, String tcp6ConnectionsFile)
   at System.Net.NetworkInformation.LinuxIPGlobalProperties.GetActiveTcpConnections()
   at Elasticsearch.Net.Diagnostics.TcpStats.GetStates()
   at Elasticsearch.Net.HttpConnection.Request[TResponse](RequestData requestData)
   at Elasticsearch.Net.RequestPipeline.CallElasticsearch[TResponse](RequestData requestData)
   at Elasticsearch.Net.Transport`1.Request[TResponse](HttpMethod method, String path, PostData data, IRequestParameters requestParameters)
   --- End of inner exception stack trace ---

As I said all the services are working properly and even the Cerebro service is able to connect internally to the elasticsearch service, so it looks like an issue regarding the .net core service only.

I couldn't find any topic regarding the above issue so I hope someone here can help me to figure it out.

PS: the .NET Core API, deployed locally on a Windows machine (Kestrel web server), runs properly connecting to the database and elasticsearch with both of them running on Docker.

Update:

I forgot to mention that the machine is running behind a corporate proxy. Keeping that in mind, I wanted to check if my API service was able to reach the Elasticsearch service endpoint: http://projelastic:9200. To do that I executed an interactive bash shell on my API service container: docker exec -it proj-core-api /bin/bash. Here I noticed that my containers do not have access to internet

root@8e7c81f98455:/app# apt-get update
Err:1 http://security.debian.org/debian-security buster/updates InRelease
  Could not connect to security.debian.org:80 (111: Connection refused)

So I added to the docker-compose file the variables for the proxy server including the no_proxy one for the Elasticsearch service domain.

projapi:
    image: proj/projcoreapp 
    depends_on:
       - projmssql
    container_name: proj-core-api
    ports:
      - "8080:80"
    restart: always
    #Specify Environment Variables for the Api Service
    environment: 
      - ASPNETCORE_ENVIRONMENT=Docker
      - http_proxy=http://user:pwd@address      
      - https_proxy=http://user:pwd@address
      - no_proxy=projelastic

With the above change I was able to install curl in my API container and perform my initial test:

root@0573a44a6836:/app# curl -X GET http://projelastic:9200
{
  "name" : "es01",
  "cluster_name" : "proj-docker-cluster",
  "cluster_uuid" : "j7mdpxkrSRKxpHnktvnZIw",
  "version" : {
    "number" : "7.10.1",
    "build_flavor" : "default",
    "build_type" : "docker",
    "build_hash" : "1c34507e66d7db1211f66f3513706fdf548736aa",
    "build_date" : "2020-12-05T01:00:33.671820Z",
    "build_snapshot" : false,
    "lucene_version" : "8.7.0",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}

Anyway adding the proxy variables didn't fix the initial error, so as a final test I brought my source code on a Windows 10 machine where it's running a Docker version 20.10.7 and no proxy. I built the API image and run the docker-compose. All services work properly, so I still think that on the Linux machine the proxy is playing an important role on my current issue.

Update 2: I was able to fix that issue thanks to the @AndrewSilver suggestion. Many thanks also to @kha for providing more details.

Recommended topics

Hot tags