"java.net.BindException: Address already in use" when trying to do rapid Socket creation and destruction for load testing

Question

I'm trying to load test a Java server by opening a large number of socket connections to the server, authenticating, closing the connection, then repeating. My app runs great for awhile but eventually I get:

java.net.BindException: Address already in use: connect

According to documentation I read, the reason for this is that closed sockets still occupy the local address assigned to them for a period of time after close() was called. This is OS dependent but can be on the order of minutes. I tried calling setReuseAddress(true) on the socket with the hopes that its address would be reusable immediately after close() was called. Unfortunately this doesn't seem to be the case.

My code for socket creation is:

Socket socket = new Socket();
socket.setReuseAddress(true);
socket.connect(new InetSocketAddress(m_host, m_port));

But I still get this error:

java.net.BindException: Address already in use: connect after awhile.

Is there any other way to accomplish what I'm trying to do? I would like to for instance: open 100 sockets, close them all, open 200 sockets, close them all, open 300, etc. up to a max of 2000 or so sockets.

Any help would be greatly appreciated!

Malonis

Is it the server or the client that throws the exception? Does it happen after a consistent number of connections? – Kight

You probably need to change the TCP settings of the machine, see docs.alfresco.com/4.0/topic/com.alfresco.enterprise.doc/tasks/… – Willdon

Answer 1

You are exhausing the space of outbound ports by opening that many outbound sockets within the TIME_WAIT period of two minutes. The first question you should ask yourself is does this represent a realistic load test at all? Is a real client really going to do that? If not, you just need to revise your testing methodology.

BTW SO_LINGER is the number of seconds the application will wait during close() for data to be flushed. It is normally zero. The port will hang around for the TIME_WAIT interval anyway if this is the end that issued the close. This is not the same thing. It is possible to abuse the SO_LINGER option to patch the problem. However that will also cause exceptional behaviour at the peer and again this is not the purpose of a test.

Linseed

You are correct in that no real client will be opening that many connections. In my application, real clients will only open 1 TCP connection to the server. What I'm trying to do is simulate real clients and I would like to have as many simulated clients running on the same machine as possible. – Malonis

Your response helped me too! :-) Something I would like to add up here is that if you use the setSoLinger() method, it causes problems for the already connected peers and they sometimes get disconnected too! Don't really know what's causing this behavior but it took me a good 45 mins to figure it out! – Callboard

@M2X SO_LINGER off is the default, which causes close() to behave as we all now and love. If you set it 'on' and with a zero timeout, it causes close() and shutdown() to issue a reset instead of a proper close. I didn't even want to mention this because it is such an undesirable behaviour. If there are any valid reasons to mess around with SO_LINGER in a normal TCP/IP application I have yet to encounter them ... in nearly thirty years. – Linseed

@rogerdpack 'Reassigns sockets' is meaningless. There is no such operation. I have no idea what you are asking here. – Linseed

Answer 2

Not using bind() but setReuseAddress(true) is just weird, I hope you do understand the implications of setReuseAddress (and the point of). 100-2000 is not a great number of sockets to open, however the server you are attempting to connect to (since it looks the same addr/port pair), may just drop them w/ a normal backlog of 50.

Edit: if you need to open multiple sockets quickly (ermm port scan?), I'd very strongly recommend using NIO and connect()/finishConnect() + Selector. Opening 1000 sockets in the same thread is just plain slow. Forgot you may need finishConnect() either way in your code.

Invite

Answer 3

I think that you should plan on the port you want to use to connect to be in use. By that I mean try to connect using the given port. If the connect fails (or in your case throws an exception), try to open the connection using the next port number.

Try wrapping the connect statement in a try/catch.

Here's some pseudo-code that conveys what I think will work:

portNumber = x //where x is the first port number you will try
numConnections = 200 // or however many connections you want to open
while(numConnections > 0){
    try{
        connect(host, portNumber)
        numConnections--
    }catch(){}
    portNumber++
}

This code doesn't cover corner cases such as "what happens when all ports are in use?"

Mallorymallow

This doesn't make any sense. He is trying to connect to a service that is listening at a fixed port number, not one that can be incremented, and his problem is the local port space, not the remote port space. – Linseed

something like this might help, if you're truly desperate: got a problem with that port still being in use? Try again! :) – Presbyter

@Presbyter It would only help if you didn't know the target port, and if you don't know that you don't know nothing. The code presented here is a port scanner, not in any sense a conventional TCP client. This answer has nothing to do with the question that was asked, whatsoever. – Linseed