How can I fix "kex_exchange_identification: read: Connection reset by peer"?

Question

I want to copy data with scp in a GitLab pipeline using PRIVATE_KEY.

The error is:

kex_exchange_identification: read: Connection reset by peer
Connection reset by x.x.x.x port 22
lost connection

Pipeline log:

$ mkdir -p ~/.ssh

$ echo "$SSH_PRIVATE_KEY" | tr -d '\r' > ~/.ssh/id_rsa

$ chmod 600 ~/.ssh/id_rsa

$ eval "$(ssh-agent -s)"
Agent pid 22

$ ssh-add ~/.ssh/id_rsa
Identity added: /root/.ssh/id_rsa (/root/.ssh/id_rsa)

$ ssh-keyscan -H $IP >> ~/.ssh/known_hosts
# x.x.x.x:22 SSH-2.0-OpenSSH_7.2p2 Ubuntu-4ubuntu2.10
# x.x.x.x:22 SSH-2.0-OpenSSH_7.2p2 Ubuntu-4ubuntu2.10

$ scp -rv api.yml root@$IP:/home/services/test/

Executing: program /usr/bin/ssh host x.x.x.x, user root, command scp -v -r -t /home/services/test/

OpenSSH_8.6p1, OpenSSL 1.1.1l  24 Aug 2021
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: Authenticator provider $SSH_SK_PROVIDER did not resolve; disabling
debug1: Connecting to x.x.x.x [x.x.x.x] port 22.
debug1: Connection established.
debug1: identity file /root/.ssh/id_rsa type -1
debug1: identity file /root/.ssh/id_rsa-cert type -1
debug1: identity file /root/.ssh/id_dsa type -1
debug1: identity file /root/.ssh/id_dsa-cert type -1
debug1: identity file /root/.ssh/id_ecdsa type -1
debug1: identity file /root/.ssh/id_ecdsa-cert type -1
debug1: identity file /root/.ssh/id_ecdsa_sk type -1
debug1: identity file /root/.ssh/id_ecdsa_sk-cert type -1
debug1: identity file /root/.ssh/id_ed25519 type -1
debug1: identity file /root/.ssh/id_ed25519-cert type -1
debug1: identity file /root/.ssh/id_ed25519_sk type -1
debug1: identity file /root/.ssh/id_ed25519_sk-cert type -1
debug1: identity file /root/.ssh/id_xmss type -1
debug1: identity file /root/.ssh/id_xmss-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_8.6
kex_exchange_identification: read: Connection reset by peer
Connection reset by x.x.x.x port 22
lost connection

Extravert

Came across this question when ssh an Asustor NAS. Check /usr/builtin/etc/ipblock/ipblock.deny to see if any IP is blocked. I think this is somewhat equivalent to /etc/hosts.deny. – Deidredeific

Is this a newer problem (e.g., with particular versions of some software? OpenSSH?)?—Some error message at Super User (2019), Unix & Linux (2019), Super User (2020), Nixcraft (2021), Ask Ubuntu (2022). – Cooker

That is, is there an older canonical question? The answers don't offer any version information. – Cooker

Is it specific to GitLab? The answers are all unspecific; it isn't specified if it was actually on GitLab or not. – Cooker

A server reboot fixed it for me – Aculeate

Answer 1

kex_exchange_identification: read: Connection reset by peer

When an SSH client connects to an SSH server, the server starts by sending a version string to the client. The error that you're getting means that the TCP connection from the client to the server was "abnormally closed" while the client was waiting for this data from the server, in other words immediately after the TCP connection was opened.

As a practical matter, it's likely to mean one of two things:

1. The SSH server process malfunctioned (crashed), or perhaps it detected some serious issue causing it to exit immediately.
2. Some firewall is interfering with connections to the ssh server.

It looks like the ssh-keyscan program was able to connect to the server and get a version string without an error. So the SSH server process is apparently able to talk to a client without crashing.

You should talk the administrators of this x.x.x.x host and the network that it's attached to, to see if they can identify the problem from their end. It's possible that something—a firewall, or the ssh server process itself—is seeing the multiple connections, first from the ssh-keyscan process, then by the scp program, as an intrusion attempt. And it's blocking the second connection attempt.

Downtime

Your comment saved me after 1 hour of struggling to figure it out. It was the VPN that I was connected to. Now everything works fine. – Switzerland

Thanks for this, exactly what happened in my case. I had to reboot the server b/c it completely locked me out. That helped though. – Hanrahan

Additionally, if the SSH error is intermittent it could be due to some limit rule reached on a firewall. Throttleling/lowering the rate of connections might make it work. – Contracted

for me this was, enabling the VPN, it did the trick. I wasn't able to connect to SSH using non-VPN connection – Aspergillosis

Thanks, my issue was Avast antivirus blocking the connection. – Anette

Answer 2

I had the same problem. I rebooted the server, then it was all good.

Sappanwood

Your answer could be improved with additional supporting information. Please edit to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers in the help center. – Partlet

same here after updating from ubuntu 22.04 to 22.10. all i had to do is reboot – Linchpin

What kind of server was that? Operating system, edition, version, etc.? Remote or local? Virtual or real hardware? – Cooker

The same problem here, trying to ssh into a raspberry pi (DietPi distro). I rebooted the RPi and ssh started to work again. – Drislane

Also worked on an LXC container. – Agna

how did you reboot the server? I'm getting this error while pushing my code to Gitlab – Coadjutor

wrong address and port caused this problem for me – Poultry

in my situation it was not only SSH connection problems, but also on port 80 and 81. Reboot solved problem. Before facing issues, I was tunneling connections to remote machine using other remote machine. Then I configured WiFi network on issued machine. Maybe that was the cause, that another NIC started working and another IP addr became available. – Golliner

Probably the service sshd was crashed. Next time try: sudo service sshd restart – Iguana

Answer 3

TL;DR:
find the server-side process listen-to-ssh port and kill it, and then restart the ssh service. It should solve this problem.

On the client side:

ssh account@ip -pPORT

kex_exchange_identification: read: Connection reset by peer

I tried it on the server side:

service ssh status

[ ok ] sshd is running.

service ssh restart

[ ok ] Restarting OpenBSD Secure Shell server: sshd.

but the client side ssh command still fail with the same kex_exchange_identification error.

Then I stop the ssh service on the server side (as root):

service ssh stop

[ ok ] Stopping OpenBSD Secure Shell server: sshd.

And the following client side ssh command still fails with the same kex_exchange_identification error. It's strange; if no process listen the port, it should be the error Connection refused.

It could be the process on the server side listening to the SSH port is dead, and even a restart / stop service do not work. So to find the process, and killing it may solve the problem.

The PORT here is the SSH port defined in 'server /etc/ssh/sshd_config', and the default is 22. As root:

netstat -ap | grep PORT

tcp 0 0 0.0.0.0:PORT 0.0.0.0:* LISTEN 8359/sshd
tcp6 0 0 [::]:PORT [::]:* LISTEN 8359/sshd

kill 8359
netstat -ap | grep PORT

no result

service ssh start

[ ok ] Starting OpenBSD Secure Shell server: sshd.

netstat -ap | grep PORT

tcp 0 0 0.0.0.0:PORT 0.0.0.0:* LISTEN 31418/sshd: /usr/sb
tcp6 0 0 [::]:PORT [::]:* LISTEN 31418/sshd: /usr/sb

The following client-side ssh command succeed.

Panne

Answer 4

I met this issue after I changed my Apple ID password, so I updated my Apple ID and restarted my Mac. It works now.

git pull origin master

Output:

kex_exchange_identification: read: Connection reset by peer
Connection reset by 20.205.243.166 port 22
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

Joceline

Restarting fixed that issue indeed. – Tananarive

Was it on GitLab? Or somewhere else? GitHub? – Cooker

Answer 5

I was having the same issue and resolved by:

1. Check if sshd is running on both machine.

2. Check if there exists any firewall rules which is not letting you succeed.

   iptables -L

3. check packets received while you are doing ssh using tcpdump on server:

   tcpdump -i any dst destination_IP and src Source_IP and dst port 22

output will show only 3 packets are being shared and the server closed connection.

4. check if hosts.allow is allowing only some host to make sshd connection and denying all other.

   cat /etc/hosts.allow

output may include: sshd: IP-range subnet_range

Also, check if hosts.deny is denying some ip range to make sshd connection:

cat /etc/hosts.deny

Output may include: sshd: ALL

If point 4 is your cause, then add your machine IP (from where you are trying to make ssh connection) as a hosts allowed on server machine. Follow these steps, then, you are good to go.

Dividivi

Answer 6

I suggest to check the routing table for one possibility. In my case on Ubuntu 20.04 (Focal Fossa), I added a local network routing entry to recover when I got the same error message when connecting to the server using SSH. It had disappeared unexpectedly, leaving only the default route.

route -n Kernel IP routing table Destination     Gateway

Output:

Genmask         Flags Metric Ref    Use Iface
0.0.0.0         192.168.1.1     0.0.0.0         UG    0      0        0 enp1s0
192.168.1.0     0.0.0.0         255.255.255.0   U     0      0        0 enp1s0  # <= disappeared

It seemed as if ack was being filtered by an incomplete routing table although the first syn passed.

Katheykathi

There is a similar problem with routing. The initialization of the connection went over the WAN, and the response over the LAN (Vlan) – Takakotakakura

Answer 7

I was facing the same issue whenever I was trying to connect to my Amazon Linux 2 instance through SSM or terminal:

Why did it show me the error: The /etc/hosts.deny file specifies which IP addresses are not permitted to connect to the host. Where as I had added "ALL: ALL" to /etc/hosts.deny and it will deny all IP address to not connect to the instance (server).

Solution: Simply you can stop the instance and edit the USER DATA and run the command

sed -i 's/ALL: ALL/#ALL: ALL/g' /etc/hosts.deny

This will comment hosts.deny. Once this is done, start the instance (server).

Widget

Answer 8

Similar to naoki-ogawa, I had a problem with my routing table. In my case, I had an extra route for my local network.

As root:

route

Output:

Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
default         RT-AX92U-3E20   0.0.0.0         UG    100    0        0 eno1
link-local      0.0.0.0         255.255.0.0     U     1000   0        0 virbr1
192.168.50.0    RT-AX92U-3E20   255.255.255.0   UG    10     0        0 eno1
192.168.50.0    0.0.0.0         255.255.255.0   U     100    0        0 eno1

I simply removed the gateway on the local network (192.168.50.0):

route del 192.168.50.0/24 via 192.168.50.1

The problem was resolved.

Tiger

Answer 9

For those who came across this page after upgrading a FreeBSD machine to 13.1 and then trying to ssh into it, see Bug 263489. sshd does not work after reboot to 13.1-RC4.

After the upgrade, the previous sshd daemon (OpenSSH < 8.2) is still running with new configurations (OpenSSH >= 8.2). The solution is to stop and then restart the sshd daemon. The FreeBSD 13.1 release notes now mention this and after 13.1, the freebsd-update script will automatically restart the daemon.

Pledgee

Answer 10

On CentOS 7.9.2009 create empty dir

mkdir /var/empty/sshd
chmod 0711 /var/empty/sshd

This helped to me

Thearchy

Thank you for your interest in contributing to the Stack Overflow community. This question already has quite a few answers—including one that has been extensively validated by the community. Are you certain your approach hasn’t been given previously? If so, it would be useful to explain how your approach is different, under what circumstances your approach might be preferred, and/or why you think the previous answers aren’t sufficient. Can you kindly edit your answer to offer an explanation? – Rigorous

Answer 11

I had this error today when I was trying to use my Dell laptop running Ubuntu 20.04.5 LTS (Focal Fossa) and trying to SSH into a Raspberry Pi. When I was on my home Wi-Fi network and tried to SSH into the pi (also on my home Wi-Fi network) I got the error:

ssh [email protected]

Output:

kex_exchange_identification: read: Connection reset by peer'

However, when I switched my Ubuntu Laptop over to a mobile hotspot, the error disappeared, and I was able to SSH without issue. Will update this post as soon as I figure out how to resolve the root cause.

---

Issue resolved (but full reason unclear). I followed the instructions to change my DNS server here to 8.8.8.8 and 8.8.4.4.

After about 5 minutes had elapsed, I was able to use SSH from my command line terminal just fine.

Burnight

What is the Pi running? I get this on Ubuntu 20.04 and my Pi 3b. My Raspbian Pi 3b devices never have this issue. I let it cool down and try again later and it works fine. – Magallanes

Answer 12

I had the same error. I have an Ubuntu 20.04 host and two RHEL 8 VMs running on VMware. I log in into the two VMs from my Ubuntu terminal. I use Ethernet and Wi-Fi connections. Every time I tried to log into a VM after rebooting it, I would get the error:

kex_exchange_identification: read: Connection reset by peer
Connection reset by x.x.x.x port 22

Restarting the sshd service would not solve the problem. Sometimes, the problem would be resolved if I physically disconnected and reconnected the Ethernet cable.

Finally I turned off my Wi-Fi connection with:

nmcli conn down <name_of_Wi-Fi_connection>

Or turning it off from settings and this gave me a permanent solution.

Both my Ethernet and Wi-Fi connections (static connections) had the same IP address, so I think the VMs were rejecting two "suspicious" similar connections.

Elielia

Please review Why not upload images of code/errors when asking a question? (e.g., "Images should only be used to illustrate problems that can't be made clear in any other way, such as to provide screenshots of a user interface.") and do the right thing (it covers answers, program input, and program output as well). Thanks in advance. – Cooker

What name? The SSID? Or something else? – Cooker

Answer 13

In my case it was with a Synology server, the private IP accidentally went into the DSM's block list. Removing it and putting the whole subnet on the allow list fixed the issue.

Sipple

Answer 14

I had to change my DNS, I used shecan.ir If you are from Iran, you most probably have the same issue, particularly when using gitlab.com

Wakashan

Answer 15

Maybe useful for those using proxy like myself:

I encountered the same issue when uploading files in the VSCode terminal. However, executing the same command in the terminal of the local machine fixed this issue.

According to Google, when directly accessing HTTP/HTTPS services using tools like VSCode for port forwarding within the development machine, proxy issues may arise.

Monaural

Answer 16

Maybe you should change the resolution of host

Press win+R, enter C:\Windows\System32\drivers\etc to open the location of the host file:

add [IP] : Domain name for example 1.1.1.1 google

Hutchins

Answer 17

The same issue with me:

I have fixed the issue by doing the below steps.

1. edit file etc/hosts.allow. Command to do so sudo nano /etc/hosts.allow.
2. At the end, update the value of ALL keys to ALL like ALL : ALL. Save the file and try again.

Basically, ALL might be set to something else therefore while establishing ssh connection to the host, it is expecting that the request should come from the IP address starting from 10...* if ALL set to ALL : 10.. Therefore by replacing 10. with ALL, you are allowing connection from everywhere.

Buckjumper

Answer 18

Try to check if OpenSSH server is up and running on the server side.

Try checking the sshd configuration. It worked this way for me.

Rhymester

As it’s currently written, your answer is unclear. Please edit to add additional details that will help others understand how this addresses the question asked. You can find more information on how to write good answers in the help center. – Quiff

This is a very simplistic answer and rather unhelpful. – Pledgee