RequestHandlerRetryAdvice cannot be made to work with Ftp.outboundGateway in Spring Integration

Question

My situation is similar to the one described in this SO question. The difference being that I don't use a WebFlux.outboundGateway but an Ftp.outboundGateway on which I call an AbstractRemoteFileOutboundGateway.Command.GETcommand, the common problem being that I can't get the defined RequestHandlerRetryAdvice to be used.

The configuration looks like this (stripped down to the relevant parts):

@RestController
@RequestMapping( value = "/somepath" )
public class DownloadController
{
   private DownloadGateway downloadGateway;

   public DownloadController( DownloadGateway downloadGateway )
   {
      this.downloadGateway = downloadGateway;
   }

   @PostMapping( "/downloads" )
   public void download( @RequestParam( "filename" ) String filename )
   {
      Map<String, Object> headers = new HashMap<>();

      downloadGateway.triggerDownload( filename, headers );
   }
}    

@MessagingGateway
public interface DownloadGateway
{
   @Gateway( requestChannel = "downloadFiles.input" )
   void triggerDownload( Object value, Map<String, Object> headers );
}

@Configuration
@EnableIntegration
public class FtpDefinition
{
   private FtpProperties ftpProperties;

   public FtpDefinition( FtpProperties ftpProperties )
   {
      this.ftpProperties = ftpProperties;
   }

   @Bean
   public DirectChannel gatewayDownloadsOutputChannel()
   {
      return new DirectChannel();
   }

   @Bean
   public IntegrationFlow downloadFiles( RemoteFileOutboundGatewaySpec<FTPFile, FtpOutboundGatewaySpec> getRemoteFile )
   {
      return f -> f.handle( getRemoteFile, getRetryAdvice() )
                   .channel( "gatewayDownloadsOutputChannel" );
   }

   private Consumer<GenericEndpointSpec<AbstractRemoteFileOutboundGateway<FTPFile>>> getRetryAdvice()
   {
      return e -> e.advice( ( (Supplier<RequestHandlerRetryAdvice>) () -> {
         RequestHandlerRetryAdvice advice = new RequestHandlerRetryAdvice();
         advice.setRetryTemplate( getRetryTemplate() );
         return advice;
      } ).get() );
   }

   private RetryTemplate getRetryTemplate()
   {
      RetryTemplate result = new RetryTemplate();

      FixedBackOffPolicy backOffPolicy = new FixedBackOffPolicy();
      backOffPolicy.setBackOffPeriod( 5000 );

      result.setBackOffPolicy( backOffPolicy );
      return result;
   }

   @Bean
   public RemoteFileOutboundGatewaySpec<FTPFile, FtpOutboundGatewaySpec> getRemoteFile( SessionFactory sessionFactory )
   {
      return 
         Ftp.outboundGateway( sessionFactory,
                              AbstractRemoteFileOutboundGateway.Command.GET,
                              "payload" )
            .fileExistsMode( FileExistsMode.REPLACE )
            .localDirectoryExpression( "'" + ftpProperties.getLocalDir() + "'" )
            .autoCreateLocalDirectory( true );
   }

   @Bean
   public SessionFactory<FTPFile> ftpSessionFactory()
   {
      DefaultFtpSessionFactory sessionFactory = new DefaultFtpSessionFactory();
      sessionFactory.setHost( ftpProperties.getServers().get( 0 ).getHost() );
      sessionFactory.setPort( ftpProperties.getServers().get( 0 ).getPort() );
      sessionFactory.setUsername( ftpProperties.getServers().get( 0 ).getUser() );
      sessionFactory.setPassword( ftpProperties.getServers().get( 0 ).getPassword() );
      return sessionFactory;
   }
}

@SpringBootApplication
@EnableIntegration
@IntegrationComponentScan
public class FtpTestApplication {

    public static void main(String[] args) {
        SpringApplication.run( FtpTestApplication.class, args );
    }
}

@Configuration
@PropertySource( "classpath:ftp.properties" )
@ConfigurationProperties( prefix = "ftp" )
@Data
public class FtpProperties
{
   @NotNull
   private String localDir;

   @NotNull
   private List<Server> servers;

   @Data
   public static class Server
   {
      @NotNull
      private String host;

      @NotNull
      private int port;

      @NotNull
      private String user;

      @NotNull
      private String password;
   }
}

The Controller is mostly just there for testing purposes, in the actual implementation there's a poller. My FtpProperties hold a list of servers because in the actual implementation, I use a DelegatingSessionFactory to pick an instance based on some parameters.

According to Gary Russell's comment, I'd expect a failed download to be retried. But if I interrupt a download server-side (by issuing "Kick user" in a FileZilla instance), I just get an immediate stack trace and no retries:

org.apache.commons.net.ftp.FTPConnectionClosedException: FTP response 421 received.  Server closed connection.
[...]

I also need to upload files, for which I use an Ftp.outboundAdapter. In this case and with the same RetryTemplate, if I interrupt an upload server-side, Spring Integration performs two more attempts with a delay of 5s each, and only then logs java.net.SocketException: Connection reset, all as expected.

I tried to debug a little and noticed that right before the first attempt to upload through the Ftp.outboundAdapter, a breakpoint on RequestHandlerRetryAdvice.doInvoke() is hit. But when downloading through the Ftp.outboundGateway, that breakpoint is never hit.

Is there a problem with my configuration, could someone get the RequestHandlerRetryAdvice to work with Ftp.outboundGateway/AbstractRemoteFileOutboundGateway.Command.GET?

Secco

Answer 1

Sorry for the delay; we are at SpringOne Platform this week.

The problem is due to the fact that the gateway spec is a bean - the gateway ends up being initialized before the advice is applied.

I changed your code like this...

@Bean
public IntegrationFlow downloadFiles(SessionFactory<FTPFile> sessionFactory) {
    return f -> f.handle(getRemoteFile(sessionFactory), getRetryAdvice())
            .channel("gatewayDownloadsOutputChannel");
}

...

private RemoteFileOutboundGatewaySpec<FTPFile, FtpOutboundGatewaySpec> getRemoteFile(SessionFactory<FTPFile> sessionFactory) {
    return Ftp.outboundGateway(sessionFactory,
            AbstractRemoteFileOutboundGateway.Command.GET,
            "payload")
            .fileExistsMode(FileExistsMode.REPLACE)
            .localDirectoryExpression("'/tmp'")
            .autoCreateLocalDirectory(true);
}

...and it worked.

It's generally better to not deal with Specs directly and just have them inlined in the flow definition...

@Bean
public IntegrationFlow downloadFiles(SessionFactory<FTPFile> sessionFactory) {
    return f -> f.handle(Ftp.outboundGateway(sessionFactory,
            AbstractRemoteFileOutboundGateway.Command.GET,
            "payload")
            .fileExistsMode(FileExistsMode.REPLACE)
            .localDirectoryExpression("'/tmp'")
            .autoCreateLocalDirectory(true), getRetryAdvice())
        .channel("gatewayDownloadsOutputChannel");
}

Wallywalnut

Fantastic, you saved my day, thanks a ton! Being both a newbie to Spring in general and Spring Integration in particular, I would never have found that cause and solution. – Sonstrom