Displaying paginated data on client from multiple data sources from server

Question

We have 3 products from which user can order.

Eg:

• Itunes music
• Itunes Books
• AppleStore Apps

Given:

3 microservices, which gives 3 individual paginated responses for order history based on timestamp.

Need:

To display order history from all 3 sources in descending order of the timestamp and paginate them as user scrolls. i.e., sort and merge responses from 3 sources based on timestamp and display.

It is possible that user has purchased:

1. From all 3 services
2. From any 2
3. From any 1
4. None

Also, the timeline of purchase can be skewed as well. eg: The user has purchased 10 songs in Itunes music alone.

Approaches I can think of:

1. Heavy-lifting on Client Only:

Will call all 3 API's parallelly, wait until all the API's respond. Store them locally.

Merge and sort based on timestamp.

Clip the topmost n items and display on the device.

As and when user scrolls, will check if there is any data present locally and call APIs accordingly.

2. Heavy-lifting on the Server side only:

Have a proxy which will talk to all the services.

Upon receiving request from client fetchData (noOfItems, fetchedUntilTimestamp),

proxy shall fetch data from different sources by calling getData(noOfItems, fetchFromTimeStamp) for each one of them.

Each data source shall Fetch noOfItems starting with timestamp fetchFromTimeStamp and below and returns a list of data.

Proxy shall:

1. SORTED_LIST = sort list of data from multiple sources based on timestamp
2. RESULT_LIST = First noOfItems of the SORTED_LIST
3. fetchedUntilTimestamp = Timestamp of Last item of RESULT_LIST

Proxy shall return to client: RESULT_LIST + fetchedUntilTimeStamp

A client, from next request onwards, shall call fetchData (noOfItems, fetchedUntilTimestamp) with the fetchedUntilTimeStamp received from a server from the previous call

Which among the two would be preferred? Any better approach to solving this?

Mccaffrey

I created an answer under a duplicate question: softwareengineering.stackexchange.com/questions/362585/… – Africah

Answer 1

It depends on your environment.

If server resource is expensive, you should choose solution 1. Also note server not need to be called three times. connection is also expensive resource, and three connections has more exception branch. server should provide a interface to return all three kind of items.

Else you should choose Solution 2. Especially there is multiple clients, need multiple developer, implement in client means need multiple work, and may have multiple bugs. The implementation details may differ. All this need more test.

At last, In practice, the solution may decided by the bigger voice.

Jobye

Answer 2

The only advantage to Solution 1 is that you don't have to create a new facade service to call 3 micro services. The disadvantage is that your client code becomes far more complex, and you'll have to release a new client if you add a 4th product.

Solution 2 makes for a much simpler client. You can easily add a 4th product (or remove a product). You can possibly optimize performance better on the server than client.

I always try to keep specific business knowledge (how many products do we sell?) out of the client and on the server.

Englishism