How to send multiple asynchronous requests to different web services?

Question

I need to send multiple requests to many different web services and receive the results. The problem is that, if I send the requests one by one it takes so long as I need to send and process all individually.

I am wondering how I can send all the requests at once and receive the results.

As the following code shows, I have three major methods and each has its own sub methods. Each sub method sends request to its associated web service and receive the results;therefore, for example, to receive the results of web service 9 I have to wait till all web services from 1 to 8 get completed, it takes a long time to send all the requests one by one and receive their results.

As shown below none of the methods nor sub-methods are related to each other, so I can call them all and receive their results in any order, the only thing which is important is to receive the results of each sub-method and populate their associated lists.

private List<StudentsResults> studentsResults = new ArrayList();
private List<DoctorsResults> doctorsResults = new ArrayList();
private List<PatientsResults> patientsResults = new ArrayList();

main (){
    retrieveAllLists();
}

retrieveAllLists(){

     retrieveStudents();
     retrieveDoctors();
     retrievePatients();
}

retrieveStudents(){

    this.studentsResults = retrieveStdWS1();   //send request to Web Service 1 to receive its  list of students
    this.studentsResults = retrieveStdWS2();  //send request to Web Service 2 to receive its  list of students
    this.studentsResults = retrieveStdWS3(); //send request to Web Service 3 to receive its  list of students

}

retrieveDoctors(){

   this.doctorsResults = retrieveDocWS4();   //send request to Web Service 4 to receive its list of doctors
   this.doctorsResults = retrieveDocWS5();  //send request to Web Service 5 to receive its  list of doctors
   this.doctorsResults = retrieveDocWS6(); //send request to Web Service 6 to receive its  list of doctors

}

retrievePatients(){

   this.patientsResults = retrievePtWS7();   //send request to Web Service 7 to receive its list of patients
   this.patientsResults = retrievePtWS8();  //send request to Web Service 8 to receive its list of patients
   this.patientsResults = retrievePtWS9(); //send request to Web Service 9 to receive its list of patients

}

Amboina

You might need to add some context to your question. At first glance, a Publish-Subscribe model here might have been advisable but depending on your use case, that might not be advisable. For example, do you have to wait for a particular event to execute retrieveAllLists or can you preemptively retrieve the results? How would caching the results work out for you? – Ehrlich

@Ehrlich question is updated hope it answers your quesitons – Amboina

Bit sad you let the bounty expire, I think you got plenty high quality response. – Lolly

Answer 1

That is a simple fork-join approach, but for clarity, you can start any number of threads and retrieve the results later as they are available, such as this approach.

    ExecutorService pool = Executors.newFixedThreadPool(10);
    List<Callable<String>> tasks = new ArrayList<>();
    tasks.add(new Callable<String>() {
        public String call() throws Exception {
            Thread.sleep((new Random().nextInt(5000)) + 500);
            return "Hello world";
        }

    });
    List<Future<String>> results = pool.invokeAll(tasks);

    for (Future<String> future : results) {
        System.out.println(future.get());
    }
    pool.shutdown();

UPDATE, COMPLETE:

Here's a verbose, but workable solution. I wrote it ad hoc, and have not compiled it. Given the three lists have diffent types, and the WS methods are individual, it is not really modular, but try to use your best programming skills and see if you can modularize it a bit better.

    ExecutorService pool = Executors.newFixedThreadPool(10);

    List<Callable<List<StudentsResults>>> stasks = new ArrayList<>();
    List<Callable<List<DoctorsResults>>> dtasks = new ArrayList<>();
    List<Callable<List<PatientsResults>>> ptasks = new ArrayList<>();

    stasks.add(new Callable<List<StudentsResults>>() {
        public List<StudentsResults> call() throws Exception {
            return retrieveStdWS1();
        }

    });
    stasks.add(new Callable<List<StudentsResults>>() {
        public List<StudentsResults> call() throws Exception {
            return retrieveStdWS2();
        }

    });
    stasks.add(new Callable<List<StudentsResults>>() {
        public List<StudentsResults> call() throws Exception {
            return retrieveStdWS3();
        }

    });

    dtasks.add(new Callable<List<DoctorsResults>>() {
        public List<DoctorsResults> call() throws Exception {
            return retrieveDocWS4();
        }

    });
    dtasks.add(new Callable<List<DoctorsResults>>() {
        public List<DoctorsResults> call() throws Exception {
            return retrieveDocWS5();
        }

    });
    dtasks.add(new Callable<List<DoctorsResults>>() {
        public List<DoctorsResults> call() throws Exception {
            return retrieveDocWS6();
        }

    });

    ptasks.add(new Callable<List<PatientsResults>>() {
        public List<PatientsResults> call() throws Exception {
            return retrievePtWS7();
        }

    });
    ptasks.add(new Callable<List<PatientsResults>>() {
        public List<PatientsResults> call() throws Exception {
            return retrievePtWS8();
        }

    });
    ptasks.add(new Callable<List<PatientsResults>>() {
        public List<PatientsResults> call() throws Exception {
            return retrievePtWS9();
        }

    });

    List<Future<List<StudentsResults>>> sresults = pool.invokeAll(stasks);
    List<Future<List<DoctorsResults>>> dresults = pool.invokeAll(dtasks);
    List<Future<List<PatientsResults>>> presults = pool.invokeAll(ptasks);

    for (Future<List<StudentsResults>> future : sresults) {
       this.studentsResults.addAll(future.get());
    }
    for (Future<List<DoctorsResults>> future : dresults) {
       this.doctorsResults.addAll(future.get());
    }
    for (Future<List<PatientsResults>> future : presults) {
       this.patientsResults.addAll(future.get());
    }
    pool.shutdown();

Each Callable returns a list of results, and is called in its own separate thread.
When you invoke the Future.get() method you get the result back onto the main thread.
The result is NOT available until the Callable have finished, hence there is no concurrency issues.

Celsacelsius

Thanks for your answer, I am not really good with threads, just wondering if it is safe for concurrent usage? – Amboina

Each thread runs in its own thread, and eventually generate a result. That result is passed back to the call through the Future object. It is as thread-safe as you can make it, and the preferred way of doing this – Celsacelsius

Oh, and to add credibility. I used to teach java concurrent programming at university level. – Celsacelsius

Thanks for your answer and comments, what do oyu mean by "it is as thread-safe as you can make it?" by the way whats your idea about using apache camel to solve my problem? – Amboina

he means that asking whether this is thread safe is asking if a hammer is a hammer. These examples use the creation of threads. Both his and mine (which are basically the same). So these ARE threads. Hard to explain lol. – Aeolotropic

This is a correct answer. Bounty should be awarded. – Aeri

Although Camel is a great tool for integration purposes and especially if you have profound knowledge of integration patterns it also has a set of complex use cases you will have to deal with, so espcially for this task I find it over-complicated. It is an architectural decision to use Camel, not a task selection criteria – Celsacelsius

@Aeolotropic +1 for "whether this is thread safe is asking if a hammer is a hammer" – Client

Answer 2

So just for fun I am providing two working examples. The first one shows the old school way of doing this before java 1.5. The second shows a much cleaner way using tools available within java 1.5:

import java.util.ArrayList;

public class ThreadingExample
{
    private ArrayList <MyThread> myThreads;

    public static class MyRunnable implements Runnable
    {
        private String data;

        public String getData()
        {
            return data;
        }

        public void setData(String data)
        {
            this.data = data;
        }

        @Override
        public void run()
        {
        }
    }

    public static class MyThread extends Thread
    {
        private MyRunnable myRunnable;

        MyThread(MyRunnable runnable)
        {
            super(runnable);
            setMyRunnable(runnable);
        }

        /**
         * @return the myRunnable
         */
        public MyRunnable getMyRunnable()
        {
            return myRunnable;
        }

        /**
         * @param myRunnable the myRunnable to set
         */
        public void setMyRunnable(MyRunnable myRunnable)
        {
            this.myRunnable = myRunnable;
        }
    }

    public ThreadingExample()
    {
        myThreads = new ArrayList <MyThread> ();
    }

    public ArrayList <String> retrieveMyData ()
    {
        ArrayList <String> allmyData = new ArrayList <String> ();

        if (isComplete() == false)
        {
            // Sadly we aren't done
            return (null);
        }

        for (MyThread myThread : myThreads)
        {
            allmyData.add(myThread.getMyRunnable().getData());
        }

        return (allmyData);
    }

    private boolean isComplete()
    {
        boolean complete = true;

        // wait for all of them to finish
        for (MyThread x : myThreads)
        {
            if (x.isAlive())
            {
                complete = false;
                break;
            }
        }
        return (complete);
    }

    public void kickOffQueries()
    {
        myThreads.clear();

        MyThread a = new MyThread(new MyRunnable()
        {
            @Override
            public void run()
            {
                // This is where you make the call to external services
                // giving the results to setData("");
                setData("Data from list A");
            }
        });
        myThreads.add(a);

        MyThread b = new MyThread (new MyRunnable()
        {
            @Override
            public void run()
            {
                // This is where you make the call to external services
                // giving the results to setData("");
                setData("Data from list B");
            }
        });
        myThreads.add(b);

        for (MyThread x : myThreads)
        {
            x.start();
        }

        boolean done = false;

        while (done == false)
        {
            if (isComplete())
            {
                done = true;
            }
            else
            {
                // Sleep for 10 milliseconds
                try
                {
                    Thread.sleep(10);
                }
                catch (InterruptedException e)
                {
                    e.printStackTrace();
                }
            }
        }
    }


    public static void main(String [] args)
    {
        ThreadingExample example = new ThreadingExample();
        example.kickOffQueries();

        ArrayList <String> data = example.retrieveMyData();
        if (data != null)
        {
            for (String s : data)
            {
                System.out.println (s);
            }
        }
    }
}

This is the much simpler working version:

import java.util.HashSet;
import java.util.List;
import java.util.Set;
import java.util.concurrent.Callable;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;

public class ThreadingExample
{

    public static void main(String [] args)
    {
        ExecutorService service = Executors.newCachedThreadPool();
        Set <Callable<String>> callables = new HashSet <Callable<String>> ();

        callables.add(new Callable<String>()
        {
            @Override
            public String call() throws Exception
            {
                return "This is where I make the call to web service A, and put its results here";
            }
        });

        callables.add(new Callable<String>()
        {
            @Override
            public String call() throws Exception
            {
                return "This is where I make the call to web service B, and put its results here";
            }
        });

        callables.add(new Callable<String>()
        {
            @Override
            public String call() throws Exception
            {
                return "This is where I make the call to web service C, and put its results here";
            }
        });

        try
        {
            List<Future<String>> futures = service.invokeAll(callables);
            for (Future<String> future : futures)
            {
                System.out.println (future.get());
            }
        }
        catch (InterruptedException e)
        {
            e.printStackTrace();
        }
        catch (ExecutionException e)
        {
            e.printStackTrace();
        }
    }
}

Aeolotropic

I've never used Camel. But if this descriptions is accurate "Apache Camel is messaging technology glue with routing. It joins together messaging start and end points allowing the transference of messages from different sources to different destinations. For example: JMS -> JSON, HTTP -> JMS or funneling FTP -> JMS, HTTP -> JMS, JSON -> JMS" then as long as you have an api call that will return to you a java object within the above "call" methods, then sure. – Aeolotropic

Thanks for including imports in your example :) – Cutwork

Answer 3

You can ask your jax-ws implementation to generate asynchronous bindings for the web service.

This has two advantages that I can see:

1. As discussed in Asynchronous web services calls with JAX-WS: Use wsimport support for asynchrony or roll my own? , jax-ws will generate well-tested (and possibly fancier) code for you, you need not instantiate the ExecutorService yourself. So less work for you! (but also less control over the threading implementation details)
2. The generated bindings include a method where you specify a callback handler, which may suit your needs better than synchronously get() ting all response lists on the thread calling retrieveAllLists(). It allows for per-service-call error handling and will process the results in parallel, which is nice if processing is non-trivial.

---

An example for Metro can be found on the Metro site. Note the contents of the custom bindings file custom-client.xml :

<bindings ...>    
    <bindings node="wsdl:definitions">
        <enableAsyncMapping>true</enableAsyncMapping>
    </bindings>    
</bindings>

When you specify this bindings file to wsimport, it'll generate a client which returns an object that implements javax.xml.ws.Response<T>. Response extends the Future interface that others also suggest you use when rolling your own implementation.

So, unsurprisingly, if you go without the callbacks, the code will look similar to the other answers:

public void retrieveAllLists() throws ExecutionException{
    // first fire all requests
    Response<List<StudentsResults>> students1 = ws1.getStudents();
    Response<List<StudentsResults>> students2 = ws2.getStudents();
    Response<List<StudentsResults>> students3 = ws3.getStudents();

    Response<List<DoctorsResults>> doctors1 = ws4.getDoctors();
    Response<List<DoctorsResults>> doctors2 = ws5.getDoctors();
    Response<List<DoctorsResults>> doctors3 = ws6.getDoctors();

    Response<List<PatientsResults>> patients1 = ws7.getPatients();
    Response<List<PatientsResults>> patients2 = ws8.getPatients();
    Response<List<PatientsResults>> patients3 = ws9.getPatients();

    // then await and collect all the responses
    studentsResults.addAll(students1.get());
    studentsResults.addAll(students2.get());
    studentsResults.addAll(students3.get());

    doctorsResults.addAll(doctors1.get());
    doctorsResults.addAll(doctors2.get());
    doctorsResults.addAll(doctors3.get());

    patientsResults.addAll(patients1.get());
    patientsResults.addAll(patients2.get());
    patientsResults.addAll(patients3.get());
}

If you create callback handers such as

private class StudentsCallbackHandler 
            implements AsyncHandler<Response<List<StudentsResults>>> {
    public void handleResponse(List<StudentsResults> response) {
        try {
            studentsResults.addAll(response.get());
        } catch (ExecutionException e) {
            errors.add(new CustomError("Failed to retrieve Students.", e.getCause()));
        } catch (InterruptedException e) {
            log.error("Interrupted", e);
        }
    }
}

you can use them like this:

public void retrieveAllLists() {
    List<Future<?>> responses = new ArrayList<Future<?>>();
    // fire all requests, specifying callback handlers
    responses.add(ws1.getStudents(new StudentsCallbackHandler()));
    responses.add(ws2.getStudents(new StudentsCallbackHandler()));
    responses.add(ws3.getStudents(new StudentsCallbackHandler()));

    ...

    // await completion 
    for( Future<?> response: responses ) {
        response.get();
    }

    // or do some other work, and poll response.isDone()
}

Note that the studentResults collection needs to be thread safe now, since results will get added concurrently!

Lolly

Apache Camel is an integration framework. You'd have to give a bit more background information if you want to know if it'd be worth your trouble to learn and use it. Like: * How often do the sources for your data change? Is it these services and will it still be these services next year, or do the sources come and go? * What are you writing? a client? Or a service that combines data from multiple services? Or a service bus? * Are there more steps in the chain after you have collected this information? Do you need to export to excel? Write to database? Print on the user's screen? Save to file? – Lolly

Or in short: Depends on how much more integration you are doing. If this is all, I'd not bother with Camel. If there's more, Camel may well be a very interesting option. – Lolly

Answer 4

Looking at the problem, you need to integrate your application with 10+ different webservices.While making all the calls asynchronous. This can be done easily with Apache Camel. It is a prominent framework for enterprise integration and also supports async processing. You can use its CXF component for calling webservices and its routing engine for invocation and processing results. Look at the following page regarding camel's async routing capability. They have also provided a complete example invoking webservices async using CXF, it available at its maven repo. Also see the following page for more details.

Jalbert

Answer 5

You might consider the following paradigm in which you create work (serially), but the actual work is done in parallel. One way to do this is to: 1) have your "main" create a queue of work items; 2) create a "doWork" object that queries the queue for work to do; 3) have "main" start some number of "doWork" threads (can be same number as number of different services, or a smaller number); have the "doWork" objects put add their results to an object list (whatever construct works Vector, list...).

Each "doWork" object would mark their queue item complete, put all results in the passed container and check for new work (if no more on the queue, it would sleep and try again).

Of course you will want to see how well you can construct your class model. If each of the webservices is quite different for parsing, then you may want to create an Interface that each of your "retrieveinfo" classes promises to implement.

Stonebroke

Answer 6

It has got various option to develop this.

1. JMS : quality of service and management, e.g. redelivery attempt, dead message queue, load management, scalability, clustering, monitoring, etc.
2. Simply using the Observer pattern for this. For more details OODesign and How to solve produce and consumer follow this Kodelog**

Encyclopedic

How does JMS make this asynchronous? – Quintus

Your each method goes as one JMS component – Encyclopedic

@LutzHorn whats your suggestion, Thanks @ Kumar – Amboina

Regarding the JMS , just hit the JMS Component and do your other stuff which is not dependent on the response and in the JMS after completing the response store that in the database with transaction id. So, independently you run all your services and when you get all your response back(Some kind of Observer pattern but more reliable when using with JMS should be used to collate all your response) and do the rest of the stuff. – Encyclopedic

@Amboina I feel Niels Bech Nielsen solution is perfectly fine and some sort matches my second solutions. And Hussain Pirosha is very good and totally depend on this but I feel it is your call to use third party systems to just call multiple web services asynchronous or write your know stuff to complete the Job. – Encyclopedic