What pattern can I use for partial success when calling a method which calls a proxy to aggregate data from multiple endpoints?

Question

As an example, suppose I have the following class:

public class FruitBasket {
    private List<Apple> apples;
    private List<Orange> oranges;
    // getters and setters...
}

Now suppose I also have a method somewhere which gets the FruitBasket.

public FruitBasket getFruitBasket() {
    //...
}

Now further suppose that the getFruitBasket method aggregates data from two different sources, which are accessed via a proxy. For example, there is a server AppleTree to get objects of type Apple, and a server OrangeTree to get objects of type Orange, and both are accessed via a proxy called OrchardGate. This is the reason why I want to write a single getFruitBasket method rather than getApples and getOranges, to minimise the latency when calling from my application to the OrchardGate.

In the case where Apple objects and Orange objects were retrieved successfully, there is no problem, I can just return the FruitBasket. In the case where there was a problem accessing or inside the OrchardGate, or in both the AppleTree and OrangeTree, I can also handle this by throwing a descendant of RuntimeException (or even Exception, if I add it to the getFruitBasket throws clause as appropriate).

However, what happens in the partial success case? What happens if I can access the AppleTree server fine, but I can't access the OrangeTree server due to some transport issue between the OrchardGate and OrangeTree?

As far as I can see there are only four options, and all are absolutely horrible:

• I could throw an exception, meaning that even though the Apple objects were received successfully, the FruitBasket would not be returned due to the lack of Orange objects.
• I could ignore the error and just return an empty list of Orange objects. This would mean that the client would not be able to see an error when looking for the Orange objects, instead it would just look as if no Orange objects existed on the OrangeTree server.
• I could add a field to FruitBasket called errorCodes which contained a list of the errors encountered when accessing the FruitBasket. Then I could add an error code to that list called PATH_FLOODED to signify the error I had encountered. However, this field errorCodes does not belong on the FruitBasket domain object at all. This field is not relevant to a FruitBasket, but only in the transaction to retrieve the FruitBasket.
• I could throw an exception, but attach the incomplete FruitBasket to the exception. This is also horrible because exceptions should only contain error information - they should not contain any fruit baskets.

I have described this problem in Java but I imagine this problem extends to multiple languages. I was quite surprised to not find a discussion of it already. Is there any standard pattern for writing methods which can return a partial success? Or anything I have missed?

Mucoprotein

Why don't you use an abstract Class like Fruit and use a List<Fruit> fruits within your FruitBasket . Then create class Apple and class Orange which are extends Fruit so that even if you put Orange and Apple in one List you are able to differentiate both using instance of operation. – Defalcate

did my comment solved your problem ? – Defalcate

I think you have misunderstood my question. First of all, I don't think that it's always possible to have a superclass for two object types. In fact, the reason I chose to describe the problem in terms of Apple and Orange is because of the English phrase "it's apples and oranges" meaning it's two completely different types of thing. However, even if you were able to do this, it wouldn't solve the problem, because the problem is how to represent a partial success case! – Mucoprotein

Ok sorry, my english is not so good :) . What about the idea of using boolean and then if the accessing to the Servers was successful then set the boolean value to true otherwise to false and before you trying to access the FruitBasket call the hasApples() or hasOranges() methods to make sure that these objects exists otherwise fire exceptions/errors – Defalcate

You seem to have described two conflicting approaches. In the first, a boolean value is set. Where is this boolean located? In the second, you call hasApples or hasOranges. The problem here is that each call would need to go over the remote connection to the OrchardGate. This would defeat the purpose of having an aggregate method - in fact it would be slower than calling a getApples and getOranges separately (three calls rather than two). – Mucoprotein

I think you don't understand me. I think you need an other method maybe called loadFruitBasket() which loads the Apple and Orange Objects from server. Because you have to access the servers, to get the apples and oranges you can set the boolean right then to true when the accessing was successful or to false wehen it was a failure. Then before using the getFruitBasket() you can ask the Class which had loaded the FruitBasket if it hasApples() or hasOranges() . with this you don't have to access the servers so many times and can display diffreent erros. – Defalcate

Your idea is quite sound: we would define a FruitBasketLoader class which retrieves FruitBasketData from the OrchardGate. The caller exchanges only domain objects with the FruitBasketLoader, so it provides an abstraction of the messy interface in terms of the desired API......whereas FruitBasketData does not have follow the domain model as it is abstracted away from the caller...... – Mucoprotein

.....however, I'm struggling to work out whether this is a case of setting our API by the implementation. I can't give an example without really stretching the fruit analogy, but what if later the OrangeSkin and OrangeSegments came from different servers. This is a change in the implementation but would also seemingly require us to refactor our API as we would have to replace hasOranges with hasOrangeSkin and hasOrangeSegments in order to report errors correctly. This kind of thing is usually a sign of bad design....but it's too late at night for me to fully analyse right now! – Mucoprotein

@Defalcate I agree with your approach, as I don't believe on reflection our domain model is being determined by the implementation, but rather the domain model is determined by the granularity that the client can expect to receive atomically. I can't think of a real-world example where a domain object can be non-atomically split into multiple parts where the client wouldn't want to retrieve those parts individually by default. And if an object is being atomically split (e.g. a float being split into an exponent and mantissa) then we can easily adopt an "all or nothing" approach. – Mucoprotein

@Defalcate if you post your idea as an answer, I should be able to accept it. – Mucoprotein

Answer 1

I think you need an other method maybe called loadFruitBasket() which loads the Apple and Orange Objects from server. Because you have to access the servers to get the apples and oranges, you can set the boolean right then to true when the accessing was successful or to false wehen it was a failure. Then before using the getFruitBasket() you can ask the Class which had loaded the FruitBasket if it hasApples() or hasOranges() . with this you don't have to access the servers so many times and can display diffreent erros.

Defalcate

Answer 2

I don't see anything wrong with this approach.

I could add a field to FruitBasket called errorCodes which contained a list of the errors encountered when accessing the FruitBasket. Then I could add an error code to that list called PATH_FLOODED to signify the error I had encountered. However, this field errorCodes does not belong on the FruitBasket domain object at all. This field is not relevant to a FruitBasket, but only in the transaction to retrieve the FruitBasket

FruitBasketTransaction {
  FruitBasket fruitBasket;
  List<Error> errorCodes;

  // ... constructor and getters
}

public FruitBasketTransaction getFruitBasket() {
  // ...
}

// In your application somewhere
FruitBasketTransaction transaction = getFruitBasketTransaction();
if contains(transaction.errorCodes, APPLE_ERROR) {
  // ... tell the user
}
if contains(transaction.errorCodes, ORANGE_ERROR) {
  // ... tell the user
}

giveFruitToUser(transaction.getFruitBasket());

Fablan

I don't really like the fact that the user has to have access to an object outside of its domain. I prefer the approach which Kami illustrated as this exposes a FruitBasketLoader rather than a FruitBasketTransaction which is not a domain object but more of a controller which helps with the exchange of domain objects. – Mucoprotein

@Kidburla Yeah. This approach is definitely not ideal--it's just a really simple answer. It's also an example of this antipattern: en.wikipedia.org/wiki/Poltergeist_(computer_science) – Fablan

Answer 3

A partial result, a snapshot in time. The same holds for a translation system that gathers miscellaneous online dictionaries and services, that might or might not be available. In general with controls "search further/abort."

In general there is a minimal threshold where you make the results available, say one agent delivering its contributed results.

The pattern is not that of the results, but of managing search tasks in progress, providing results. A bit of change management is due too. When using a partial result to make a fruit basket, at a later point, you may want to repeat making a fruit basket with more variation having become available.

A service provides a time line providing data packages[]:

`Service A:   (start) [pack 0] (t1) [pack 1] (t2) [pack 2] (t3) {running}
`Service B:   (start) (t1) [pack 3] (t2) (t3) [pack 4] (t4) {ended}

Which metaphor to use depends a bit: project management for instance, or in your case production.

Brownell

I'm really struggling to understand your answer. I think you may have answered a different question from the one I was trying to ask. – Mucoprotein

IF this answer doesnot trigger recognition, it certainly is not good enough. And an other answer will be better suited. – Brownell