What exactly does the -q option of netcat do?
Asked Answered
T

1

7

I have a service whose behavior I've been debugging behind a proxy. The proxy is a black box, but the service's behavior can be simulated with a trivial python program like below:

#!/usr/bin/env python

import socket
import sys
import time

PORT = int(sys.argv[1] or 50007)
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
try:
    s.bind(('', PORT))
    s.listen(1)
    conn, addr = s.accept()
    print('conn from %s' % (addr,))
    data = conn.recv(1024)
    time.sleep(1)
    conn.sendall(data)
finally:
    conn.close()
    s.close()

When I connect to this directly with netcat, I get the expected behavior, which is that after sending some arbitrary text to stdin with a newline, the server delays for a second, echoes it back, and closes the connection. However, when I put this service behind the proxy, netcat instead immediately exits with status 0, unless I give it the -q option with some nonzero number, after which it behaves the same as the direct connection.

To quote from the manpage for netcat on my system:

-q      after EOF on stdin, wait the specified number of seconds and then quit.
        If seconds is negative, wait forever.

What I'm trying to debug is what is different about the proxy connection behavior that would cause the connection behavior to differ with the -q option, and unfortunately, the man page isn't doing much to help me out. What is the "stdin" it's referring to, if it's not my input (which never receives an EOF from me)? What EOF is -q ignoring for long enough to receive the proxy'd data back from that it doesn't have to ignore when there is no proxy mediating the connection?

EDIT: On request, here are some sample invocations, although they're about as basic as it gets:

# after starting the example server on <someserver> with <someport

$ echo foo | nc <someserver> <someport>
foo
$

# the above, just with -q1 (no change)
$ echo foo | nc -q1 <someserver> <someport>
foo
$

# after starting the example server behind the blackbox proxy
$ echo foo | nc <someproxy> <someproxyport>
$

# same, but with -q
$ echo foo | nc -q1 <someproxy> <someproxyport>
foo
$
Tristichous answered 21/1, 2017 at 7:14 Comment(7)
Do you run netcat interactively or with redirected input?Micronucleus
Depending on which TCP proxy you are using, it might be that the proxy is buffering your input. Maybe nc quits before the buffer is flushed and forwarded to your Python server?Rust
To @Leon: both.Tristichous
To @dnswlt: Well, the server has its own delay, so what precisely is the difference with the proxy's buffer delay and the delay with the directly connected server? Since I don't know fundamentally what -q is changing, I don't really know how to speak with more precision about whether it might or might not be thatTristichous
(also, when I say the proxy is a black box, I really mean I have no insight into it--no access to the machine it's running on, no documentation about it, no particular way to get any insight into it other than sending packets through it and seeing what they do)Tristichous
Can you post the exact commands you're running as well as sample data and results?Hazeghi
Sure, I've done so. They are the opposite of complicated (I made this sample server with the purpose of debugging this issue, so that's as expected)Tristichous
B
10

echo foo | netcat sends EOF in all cases. What differs is the way netcat closes its network socket.

Without -q netcat sends TCP FIN right after receiving EOF, shutting down its half of TCP connection. That is not a full connection closing, just an indication that no more data will be sent. After that netcat keeps printing data from the connection until it closes.

Normal server does not handle FIN from netcat, delays, sends echo response and then shuts down the last half of connection.

The blackbox proxy seems to react on FIN from client by closing connection immediately. That might be some kind of server load optimization, which has a sense in case with lots of web browsers, for example.

With -q netcat does not send FIN until connection is closed by the server or until -q timeout ends.

Run your four cases under tcpdump -w, then load capture file in Wireshark, follow TCP stream and see the difference. Be aware that this netcat behaviour might depend on a particular version, I confirm it for Ubuntu 16.04 + netcat-openbsd 1.105.

Barnard answered 26/1, 2017 at 0:2 Comment(5)
Thanks for the TCP FIN insight, I think that explains a lot of the difference and should be helpfulTristichous
Don't trust me blindly, run tcpdump on your system anyway ;) You will gain a skill that would help you with all similar future questions.Barnard
Don't worry, I'm not, and tcpdump's already a normal part of my debugging process, I was just too mentally stuck on the application layer thanks to the rest of the context that led me hereTristichous
ncat: invalid option -- 'q'Holmium
@JPX: it might be the case with the OpenBSD netcatBarnard

© 2022 - 2024 — McMap. All rights reserved.