"curl: (52) Empty reply from server" / timeout when querying ElastiscSearch

Question

I've ran into an annoying issue with my ElasticSearch (Version 1.5.2): Queries immediately return timeout (when I used python's Requests) or

curl: (52) Empty reply from server

when I used curl.

This only happened when the expected output was large. When I sent a similar (but smaller) query, it came back just fine.

what's going on here? and how can I overcome this?

Virilism

Answer 1

This issue was caused by Elastic running out of memory: it simply can't hold all the documents in memory. Unfortunately there's no explicit error code for this case.

There are a bunch of options to work around this (besides adding more memory):

1. You can tell Elastic to not attach the source, by specifying "_source: false". The results would then just list the relevant documents (and you would need to retrieve them).
2. You could use "source filtering" to return just part of the documents, if you dont need the whole thing - that worked for me.
3. You can also just split your query into a bunch of sub-queries. not pretty, but it would do the trick.

Virilism

The aforementioned error occurs on a freshly installed ES, totally empty. With -Xms1g, -Xmx1g in jvm.options. Looks like https://mcmap.net/q/369808/-quot-curl-52-empty-reply-from-server-quot-timeout-when-querying-elastiscsearch is more suitable in my case. – Kastroprauxel

Answer 2

just open file with

sudo nano  /etc/elasticsearch/elasticsearch.yml

and replace this setting with false
# Enable security features
xpack.security.enabled: false

Imco

for running Kibana this setting need to be true – Arrington

Thank you @Er. Shahid Khan .. Solved the problem for me @ ES 8.1 as well. Now Very well able to hit the cUrl queries through terminal to ES. – Lakeishalakeland

@AdityaGoel I'm using docker-compose for elasticsearch and Kibana for both I'm using version 8.1.2 which is the latest I guess and I'm getting "curl: (52) Empty reply from server" while hitting the URL. And elasticsearch.yml is not found when I tried the above command it is creating a new one any suggestion..? btw I'm using docker on WSL2. Also this is working fine with 7.9.2 version of Elasticseach and Kibana – Apnea

Turning off Elasticsearch security solves the problem by creating a much bigger one. – Unintelligible

Answer 3

An other explanation can be making http request when ssl/security is activated on the cluster.

In this case use

curl -X GET "https://localhost:9200/_cluster/health?wait_for_status=yellow&timeout=50s&pretty" --key certificates/elasticsearch-ca.pem  -k -u elasticuser

As stated by @FanchenBao, one can read the doc about ELK with SSL.

Update 16/08/2023 - As stated by @Nokados, one can use this command, using the --cacert option, from the updated security documentation :

curl --cacert /etc/elasticsearch/certs/http_ca.crt -u elastic:yourpassword https://localhost:9200

Smitty

One can also refer to the doc regarding how to use curl with certificate. – Vins

Now according the documentation it is sudo curl --cacert /etc/elasticsearch/certs/http_ca.crt -u elastic:yourpassword https://localhost:9200. sudo is here because /etc/elasticsearch has restricted access. password is not necessary, it can be typed interactively. – Autunite

Answer 4

I meet with the same issue on Elasticsearh 8.1.3, which is the latest version. I fixed this issue by changing the following setting from true to false in the /config/elasticsearch.yml file:

# Enable security features
xpack.security.enabled: false

I installed elastic by downloading the tar file, and unzip it, then going to the folder of elasticsearch, and running the following command:

./bin/elasticsearch

The first time you run this command, it will change the elasticsearch.yml file with the following content, which means it's a default secruity setting auto generated:

#----------------------- BEGIN SECURITY AUTO CONFIGURATION -----------------------
#
# The following settings, TLS certificates, and keys have been automatically      
# generated to configure Elasticsearch security features on 01-05-2022 06:59:12
#
# --------------------------------------------------------------------------------

# Enable security features
xpack.security.enabled: true

xpack.security.enrollment.enabled: true

# Enable encryption for HTTP API client connections, such as Kibana, Logstash, and Agents
xpack.security.http.ssl:
  enabled: true
  keystore.path: certs/http.p12

# Enable encryption and mutual authentication between cluster nodes
xpack.security.transport.ssl:
  enabled: true
  verification_mode: certificate
  keystore.path: certs/transport.p12
  truststore.path: certs/transport.p12
# Create a new cluster with the current node only
# Additional nodes can still join the cluster later
cluster.initial_master_nodes: ["DaMings-MacBook-Pro.local"]

# Allow HTTP API connections from localhost and local networks
# Connections are encrypted and require user authentication
http.host: [_local_, _site_]

# Allow other nodes to join the cluster from localhost and local networks
# Connections are encrypted and mutually authenticated
#transport.host: [_local_, _site_]

#----------------------- END SECURITY AUTO CONFIGURATION -------------------------

Festoonery

sudo nano /etc/elasticsearch/elasticsearch.yml – Shrovetide

Answer 5

When running in Docker you can disable the security by setting the environment variable xpack.security.enabled to false, e.g. in docker-compose.yml:

    environment:
      - xpack.security.enabled=false
      - discovery.type=single-node

Malloch

Answer 6

This also happens whenever you are not authorized

curl -k -u elastic:password https://localhost:9200

The above did the trick for me

Wreckful

Answer 7

In version 6.2, there are more strict checking.

for example:

curl -XPUT -H'Content-Type: application/json' 'http://localhost:9200/us/user/2?pretty=1' -d '{"email" : "[email protected]", "name" : "Mary Jones","username" : "@mary"}'
curl: (52) Empty reply from server

if you remove =1:

curl -XPUT -H'Content-Type: application/json' 'http://localhost:9200/us/user/2?pretty' -d '{"email" : "[email protected]", "name" : "Mary Jones","username" : "@mary"}'
{
  "_index" : "us",
  "_type" : "user",
  "_id" : "2",
  "_version" : 1,
  "result" : "created",
  "_shards" : {
    "total" : 2,
    "successful" : 1,
    "failed" : 0
  },
  "_seq_no" : 0,
  "_primary_term" : 1
}

it works!

Pemphigus

Answer 8

In my case it was because the url scheme https:// was missing in the endpoint url.

Modestia

Answer 9

It looks like your version causes the problem. If you follow the steps below to reinstall ES and Kibana with the correct version, your problem could be solved. Be careful with running these commands, you will lose your data!

cleanup-remove and reinstall Elasticsearch

$ sudo rm -rf /etc/elasticsearch
$ sudo rm -rf /var/lib/elasticsearch
$ sudo apt-get install elasticsearch=7.10.1 
$ sudo systemctl start elasticsearch

check it's running

$ curl http://localhost:9200/

cleanup-remove and reinstall Kibana

$ sudo apt-get remove --purge kibana
$ sudo rm -rf /etc/kibana
$ sudo rm -rf /var/lib/kibana
$ sudo apt-get install kibana=7.10.1
$ sudo systemctl start kibana

Kalfas

Answer 10

This would normally function in a secured elasticsearch container:

docker exec -it elasticstack-elasticsearch-1 /bin/bash -c 'curl --user elastic:yourpassword --cacert /usr/share/elasticsearch/config/certs/elasticsearch/elasticsearch.crt -X GET "https://localhost:9200/_cluster/health?wait_for_status=yellow&timeout=50s&pretty"'

For a docker compose stack with kibana, you don't want to turn off security entirely. But let's say you want to turn of ssl on your elasticsearch container by setting these environment variables:

- xpack.security.http.ssl.enabled=false
- xpack.security.transport.ssl.enabled=false

a healthcheck in docker compose, for the elasticsearch container, would then look like this:

   healthcheck:
     test:
       [
         "CMD-SHELL",
         "curl -s --user elastic:${ELASTIC_PASSWORD} -X GET http://localhost:9200/_cluster/health?pretty | grep status | grep -q 'green'"
       ]
     interval: 10s
     timeout: 10s
     retries: 6

mind the http instead of https and not using the certificate.

you would also have to change your kibana environment variables like ELASTICSEARCH_HOSTS=http://elasticsearch:9200 to use http.

Verret

Answer 11

Quick solution for windows and docker users, simply open a bash and write this :

docker run -d -p 9200:9200 -p 9300:9300 --name <name_you_want> -e "discovery.type=single-node" -e "xpack.security.enabled=false" docker.elastic.co/elasticsearch/elasticsearch:<your_tag>

⚠ consider replacing the <>

it's gonna create you a container with the name you want (<name_you_want>) for the elasticsearch tag you want (<your_tag>) and avoid the curl: (52) error !

Roving