I had this same issue when trying to perform a listfiles from a Linux machine to a IIS server. The code worked great from my developer workstation, but would hang when running on the server specifically due to a firewall gumming up the mix.
Must do these things in order and will require you to extend FTPSClient 3.5
- connect (implicit = true, SSLContext = TLS)
- check isPositiveCompletion
- authenticate (of course)
- execPBSZ(0)
- execPROT("P")
- set boolean to indicate Skip Passive IP (custom FTPSClient class)
- set the save connection IP address (custom FTPSClient class)
- setUseEPSVwithIPv4(false)
- enterLocalPassiveMode() or enterRemotePassiveMode()
- initiateListParsing() or any list command
a.) At this point the openDataConnection will be executed, be sure to save the port being used here
b.) The PASV command is executed
c.) The _parsePassiveModeReply is executed, here you will open the socket with the IP address you used for connecting and the saved port.
- disconnect (always)
More INFO:
My issue is specific to a firewall between the Linux machine and IIS server.
The root of my problem is that in passive mode the IP address used to open the socket when doing a data connection is different that the one used to do the initial connection. So due to two issues (see below) with APACHE commons-net 3.5 it was incredibly difficult to figure out.
My solution:
Extend FTPSClient so that I could override methods _parsePassiveModeReply & openDataConnection. My parsePassiveModeReply is really just saving the port from the reply since the reply indicates what port is being used. My openDataConnection method is using the saved port and the original IP used during connection.
Problems with APACHE FTPCLient 3.5
- Data Connection does not time out (hangs) so its is not apparent
what the problem is.
- The FTPSClient class does not skip passive IP addresses. Setting
passiveNatWorkaround to true doesn't work as I expected or maybe it
doesn't skip the IP at all.
Things to pay attention to:
- When going through a firewall you must have access to the port range
defined by IIS (see configuring Microsoft IIS firewall).
- You should also ensure you have any appropriate certificates in your
keystore or the cert specified at runtime.
Add the following to your class, this is so very helpful to know what
FTP commands are being executed.
ftpClient.addProtocolCommandListener(new PrintCommandListener(new PrintWriter(System.out), true));
- Check the FTP server logs as it will tell you what is being performed
and possible why you are having problems. You should always see a
data channel opened before executing a list. Compare the results of
your application to that of what a successful curl command performs.
- Reply codes as they will indicate where a problem is occurring.
Use the curl command to verify you have connectivity, The following
is a good start and if all is well will list the contents in the root
directory.
curl -3 ftps://[user id]:[password][ftp server ip]:990/ -1 -v --disable-epsv --ftp-skip-pasv-ip --ftp-ssl --insecure
FTPSClient extended (SAMPLE CODE)
import java.io.IOException;
import java.net.Inet6Address;
import java.net.InetSocketAddress;
import java.net.Socket;
import javax.net.ssl.SSLContext;
import org.apache.commons.net.MalformedServerReplyException;
import org.apache.commons.net.ftp.FTPReply;
import org.apache.commons.net.ftp.FTPSClient;
/**
* TODO Document Me!
*/
public class PassiveFTPSClient extends FTPSClient {
private String passiveSkipToHost;
private int passiveSkipToPort;
private boolean skipPassiveIP;
/** Pattern for PASV mode responses. Groups: (n,n,n,n),(n),(n) */
private static final java.util.regex.Pattern PARMS_PAT;
static {
PARMS_PAT = java.util.regex.Pattern.compile(
"(\\d{1,3},\\d{1,3},\\d{1,3},\\d{1,3}),(\\d{1,3}),(\\d{1,3})");
}
/**
* @param b
* @param sslContext
*/
public PassiveFTPSClient(boolean b, SSLContext sslContext) {
super(b, sslContext);
}
protected void _parsePassiveModeReply(String reply) throws MalformedServerReplyException {
if (isSkipPassiveIP()) {
System.out.println( "================> _parsePassiveModeReply" + getPassiveSkipToHost());
java.util.regex.Matcher m = PARMS_PAT.matcher(reply);
if (!m.find()) {
throw new MalformedServerReplyException(
"Could not parse passive host information.\nServer Reply: " + reply);
}
try {
int oct1 = Integer.parseInt(m.group(2));
int oct2 = Integer.parseInt(m.group(3));
passiveSkipToPort = (oct1 << 8) | oct2;
}
catch (NumberFormatException e) {
throw new MalformedServerReplyException(
"Could not parse passive port information.\nServer Reply: " + reply);
}
//do nothing
} else {
super._parsePassiveModeReply(reply);
}
}
protected Socket _openDataConnection_(String command, String arg) throws IOException {
System.out.println( "================> _openDataConnection_" + getPassiveSkipToHost());
System.out.println( "================> _openDataConnection_ isSkipPassiveIP: " + isSkipPassiveIP());
if (!isSkipPassiveIP()) {
return super._openDataConnection_(command, arg);
}
System.out.println( "================> getDataConnectionMode: " + getDataConnectionMode());
if (getDataConnectionMode() != ACTIVE_LOCAL_DATA_CONNECTION_MODE &&
getDataConnectionMode() != PASSIVE_LOCAL_DATA_CONNECTION_MODE) {
return null;
}
final boolean isInet6Address = getRemoteAddress() instanceof Inet6Address;
Socket socket;
if (getDataConnectionMode() == ACTIVE_LOCAL_DATA_CONNECTION_MODE) {
return super._openDataConnection_(command, arg);
}
else
{ // We must be in PASSIVE_LOCAL_DATA_CONNECTION_MODE
// Try EPSV command first on IPv6 - and IPv4 if enabled.
// When using IPv4 with NAT it has the advantage
// to work with more rare configurations.
// E.g. if FTP server has a static PASV address (external network)
// and the client is coming from another internal network.
// In that case the data connection after PASV command would fail,
// while EPSV would make the client succeed by taking just the port.
boolean attemptEPSV = isUseEPSVwithIPv4() || isInet6Address;
if (attemptEPSV && epsv() == FTPReply.ENTERING_EPSV_MODE)
{
System.out.println( "================> _parseExtendedPassiveModeReply a: ");
_parseExtendedPassiveModeReply(_replyLines.get(0));
}
else
{
if (isInet6Address) {
return null; // Must use EPSV for IPV6
}
// If EPSV failed on IPV4, revert to PASV
if (pasv() != FTPReply.ENTERING_PASSIVE_MODE) {
return null;
}
System.out.println( "================> _parseExtendedPassiveModeReply b: ");
_parsePassiveModeReply(_replyLines.get(0));
}
// hardcode fore testing
//__passiveHost = "10.180.255.181";
socket = _socketFactory_.createSocket();
if (getReceiveDataSocketBufferSize() > 0) {
socket.setReceiveBufferSize(getReceiveDataSocketBufferSize());
}
if (getSendDataSocketBufferSize() > 0) {
socket.setSendBufferSize(getSendDataSocketBufferSize() );
}
if (getPassiveLocalIPAddress() != null) {
System.out.println( "================> socket.bind: " + getPassiveSkipToHost());
socket.bind(new InetSocketAddress(getPassiveSkipToHost(), 0));
}
// For now, let's just use the data timeout value for waiting for
// the data connection. It may be desirable to let this be a
// separately configurable value. In any case, we really want
// to allow preventing the accept from blocking indefinitely.
// if (__dataTimeout >= 0) {
// socket.setSoTimeout(__dataTimeout);
// }
System.out.println( "================> socket connect: " + getPassiveSkipToHost() + ":" + passiveSkipToPort);
socket.connect(new InetSocketAddress(getPassiveSkipToHost(), passiveSkipToPort), connectTimeout);
if ((getRestartOffset() > 0) && !restart(getRestartOffset()))
{
socket.close();
return null;
}
if (!FTPReply.isPositivePreliminary(sendCommand(command, arg)))
{
socket.close();
return null;
}
}
if (isRemoteVerificationEnabled() && !verifyRemote(socket))
{
socket.close();
throw new IOException(
"Host attempting data connection " + socket.getInetAddress().getHostAddress() +
" is not same as server " + getRemoteAddress().getHostAddress());
}
return socket;
}
/**
* Enable or disable passive mode NAT workaround.
* If enabled, a site-local PASV mode reply address will be replaced with the
* remote host address to which the PASV mode request was sent
* (unless that is also a site local address).
* This gets around the problem that some NAT boxes may change the
* reply.
*
* The default is true, i.e. site-local replies are replaced.
* @param enabled true to enable replacing internal IP's in passive
* mode.
*/
public void setSkipPassiveIP(boolean enabled) {
super.setPassiveNatWorkaround(enabled);
this.skipPassiveIP = enabled;
System.out.println( "================> skipPassiveIP: " + skipPassiveIP);
}
/**
* Return the skipPassiveIP.
* @return the skipPassiveIP
*/
public boolean isSkipPassiveIP() {
return skipPassiveIP;
}
/**
* Return the passiveSkipToHost.
* @return the passiveSkipToHost
*/
public String getPassiveSkipToHost() {
return passiveSkipToHost;
}
/**
* Set the passiveSkipToHost.
* @param passiveSkipToHost the passiveSkipToHost to set
*/
public void setPassiveSkipToHost(String passiveSkipToHost) {
this.passiveSkipToHost = passiveSkipToHost;
System.out.println( "================> setPassiveSkipToHost: " + passiveSkipToHost);
}
}
