Make netcat discard all bytes after disconnect

Question

I use netcat to run a tiny development webserver through the bash, which is able to handle one connection at once.

Netcat is started as follows (-k to make it survive multiple connections):

nc -kl 3000

A bash script processes the browser's request which is received by netcat, and builds the response, which is sent back to the browser through the same netcat instance.

Everything works as expected, but sometimes, the browser does not get the file it requested. My suspicion: If a connection is closed before the response is sent completely, the response's remainder is sent as response to the following request (it, of course, not belongs to).

Proof

1. Terminal 1 (Server): nc -kl 3000
2. Terminal 2 (simulates Browser): nc localhost 3000
3. Type hello\n in terminal 1.
4. Terminal 2 prints hello\n.
5. Do Ctrl+C in terminal 2 to end the connection.
6. Type world\n in terminal 1.
7. Run nc localhost 3000 in terminal 2 again (new connection).
8. Terminal 2 immediately shows world\n even though world\n was actually sent when no connection existed, meant as second response line in the first connection.

Required behavior: Ignore all bytes that are passed to netcat if no connection exists.

Is it possible using netcat? (I prefer a tool like netcat as it comes pre-installed on all machines.)

Traceable

could you detail how you're interacting between netcat and your script? – Moe

I'm asking the question because cf my edit (after the line) – Moe

I interact through file descriptors (variables input and output): coproc nc -kl "$port"; input="${COPROC[0]}"; output="${COPROC[1]}" – Traceable

ok, as I said, if ① using a more evolved HTTP server (like ones in python, lua, ruby, JS…) is out of the question, ② having micro disconnections is not acceptable, ③ having data output flowing from one client to the other is neither acceptable ; then you don't have much solutions, – Moe

you need to have your http server program spawn the script you want to run, so that upon disconnection, outputs will be dropped as the script gives back execution to the parent. I believe that nmap's ncat does it. Or you can write your own webserver where you'll have control over the fds, and you'll be able to make sure to flush all output when the client disconnects. – Moe

I try to minimize the effort of setting up a minimal web server. Since netcat is a well-known tool and often even comes pre-installed (as well as the bash), I chose it. If there is no other pre-installed tool worthy of consideration on a pristine linux installation (Ubuntu in my case), and ncat fits my needs, I will stick with it. For other projects I use webpack's dev server or a small JavaScript running express. – Traceable

well, python is installed per default on ubuntu, and you have a minimalist web server implementation provided, for example to serve files you just do python -m SimpleHTTPServer <port> (which becomes http.server in python 3.x) which is rather easy to customize. – Moe

That's a good point, thank you. I am playing with ncat and if it does not work, I will use Python's simple HTTP server. – Traceable

Answer 1

well, as a workaround and if you can do that, instead of running ncat using the keep-open option, just respawn it between each connection in a while loop:

while true; do
    #what you want to do with: ncat -l 3000 
done

Then each time the process will respawn, it will discard all stdin, and then you start over with the next i/o for the following process.

Of course, if respawning your bash script isn't convenient, then it might mean you're not using the right tool for the job. (you might be able to workaround that playing with fifos or using temporary fds, but that'd be overengineering to avoid writing a script in a language that'd be a better fit for the job).

---

If you're not already doing it, did you try running your script from netcat and not the other way around?

ncat -kl 3000 -c ./script.sh

it will spawn the script for each connection, on which it will redirect stdin/stdout. Then when the client will disconnect, the script will get killed and should release your input fd.

Actually, if you're serving files as an http server, you might to look up:

ncat -lk 3000 --lua-exec httpd.lua

with the httpd.lua offered with the ncat distribution

---

If you want a simple run anywhere script to do something on your system, you can write a small python script.

It is installed per default on ubuntu (and many other systems), and you have a minimalist web server implementation provided, for example to serve files you just do:

python -m SimpleHTTPServer <port>

• cf SimpleHTTPServer for py2
• or http.server in py3

it is rather easy to customize, to fit your unique needs, and will offer you a full control over your sockets and file descriptors.

Moe

This won't work as time passes without a listening socket if I use a loop. Please correct me if I am wrong. – Traceable

What do you mean? – Moe

After ending a connection, before spawning the new nc instance, nothing is listening on port 3000. – Traceable

indeed, as tiny the amount of time in between respawns is, it's still happening. Though if there is a slight chance that you're going to have a connection in that short period of time, then you might consider you're using the wrong tool for the task, and might be a sign you should write something in a more flexible language than the shell (like python, ruby, lua…). You can either write the whole netcat+script, or a netcat replacement that will drop stdout until the next connection. – Moe

"Though if there is a slight chance that you're going to have a connection in that short period of time" That is how browsers work, thus a web server must be capable of handling it. – Traceable

Well, using netcat as a webserver is like using a swiss army knife to cut an oak tree. You might succeed eventually, but it's going to be painful, slow and you might need several knifes to cut it down. Webbrowsers are designed to work with multithreaded and fast web servers, using netcat for the task will have drawbacks, and is actually NOT recommended to use in production. – Moe

Though, in the nmap version of netcat, as I shown in my edit, there is a way to spawn a process for which stdin/stdout will be local, so chances are that it will answer your request. – Moe

Thank you. I will take a closer look at nmap's netcat version to see if it fits my needs. – Traceable

actually, nmap has taken over the development of netcat, and I believe it's supposed to be the official source tree ☺ – Moe

Thanks a lot for all your great input. I just wrote a small test script and started it using ncat -c: read line; echo "a"; sleep 5; echo "b". If the client closes the connection during sleep, the next client won't be bothered by the output of echo "b". Works like a charm. – Traceable

Answer 2

Sticking with the need for "run everywhere" script, you could try processing the input line by line and detecting if an ESTABLISHED connection exists with netstat:

while read line; do 
  netstat -an | grep 3000 | grep -q ESTABLISHED; 
  ret=$?; 
  if [ "$ret" == "0" ]; then 
    echo $line; 
  else 
    # this output is just for fun, and can be removed
    echo "not sending $line" >&2; 
  fi; 
done | nc -kl 3000

This will have the limitation of being line-based and may still send data to netcat wrongly if the connection is closed just after the netstat call. The two grep calls should make sure that you dont wrongly detect your own grep.

Ive tried looking for signals returned by netcat when the connection is closed but cant see any exposed to the shell.

The use of a fifo can be used to demonstrate that netcat keeps its stdin open, even when no connection exists:

mkfifo test

nc -kl < test

# then in a different shell
for a in {0..5}; do echo hi > test; done

The for does not block, so netcat is actively reading the stdin but buffering it (and I cant see an option to turn off buffering). This means you cant really tell the difference between netcat with or without an active connection, except by looking at the network state. netstat is one of many ways of getting this state.

Stolid

I like your solution as it stops sending as soon as the connection is lost. Unfortunately, it is too slow because I need to test the connection for each byte and even then it is not 100% reliable because the last byte sent after a connection is closed by the client will go through the following connection. – Traceable