Load balancer and API Gateway confusion [closed]

Question

I have always worked on mobile technologies and now I am stepping into backend systems, more specifically systems design. I keep coming across conflicting statements for the roles of api gateway and load balancer. Googling has only returned the same half a dozen results that mostly focus on the implementations of load balancer or api gateway service provided by some famous service. I will list here all the confusing I am facing, in hope someone can clarify all of them.

Sometimes, i come across that API Gateway is the single point of communication with client devices. On the other hand, some places mention that 'request goes to load balancer, which spreads it across servers equally'. So what is correct? API Gateway receives requests or load balancer?

Other places, when I googled the topic, say that the two are totally different. I've understood that API Gateway does a lot of stuff, like SSL termination, logging, throttling, validation, etc, but it also does load balancing. So API Gateway is a load balancer itself, equipped with other responsibilities?

On the topic, I want to understand if load balancer distribute load among servers of the same cluster or across different data centers or clusters? And what about API Gateway?

What is so specific to api gateway that it is a choice by default for micro-service architecture? Where are API gateways hosted? A DNS resolves domain name to a load balancer or api gateway?

As it might be clear, I am totally confused. In what systems does a load balancer benefit more than API Gateway, if the question is correct.

Douse

Related articles blog.hubspot.com/website/api-gateway-vs-load-balancer and solo.io/topics/api-gateway/api-gateway-vs-load-balancer/…. – Claudiaclaudian

Answer 1

API gateway predominately does API management and provides various other key features such as IAM (Identity and Access Management), Rate limiting, circuit breakers. Hence, it mainly eliminates the need to implement API-specific code for functionalities such as security, caching, throttling, and monitoring for each of the microservice. Microservices typically expose the REST APIs for use in front ends, other microservices and 3rd party apps with help of API gateway.

However, normally, the API Management does not include load balancing function, so it should be used in conjunction with a load balancer to achieve the same.

In system architecture based on Azure, there is Azure Application Gateway which is a load balancer that runs on Layer 7 and provides more features than traditional load balancer ( Layer 4 ) in terms of routing traffic using routing decisions based on additional attributes of HTTP request or content of traffic. This can also be termed as an application load balancer. It shall be used in conjunction by Azure API Management (API gateway). Azure has a Traffic Manager for operating at DNS level which uses DNS to direct client requests to the most appropriate service endpoint based on a traffic-routing method and the health of the endpoints. Traffic manager also uses the rules configured at the DNS level and enables dstribution of the the load over multiple regions and data centers. Within every region or data center, there shall be application gateways coupled with load balancers such that, the application gateways shall help in determining the application server to fetch response from and the load balancer shall help in load balancing.

System overview based on Azure :

Here are few related references:

1. Azure Application Gateway - https://learn.microsoft.com/en-us/azure/application-gateway/application-gateway-introduction

2. Azure Load Balancer- https://learn.microsoft.com/en-us/azure/load-balancer/load-balancer-overview

3. Azure Traffic Manager - https://learn.microsoft.com/en-us/azure/traffic-manager/traffic-manager-overview

4. Scenario Architecture - https://learn.microsoft.com/en-us/azure/traffic-manager/traffic-manager-load-balancing-azure

Mortification

it is not correctly 100%. Where is the API Gateway here? – Affaire

@toto' It is written as AppGW in the diagram. – Schopenhauer

@Schopenhauer AppGW is not the same as API Management (or API Gateway) – Affaire

Answer 2

API Gateway and Load Balancer are 2 different things.

Load Balancer -> Its a software which works at protocol or socket level (eg. tcp, http, or port 3306 etc.) Its job is to balance the incoming traffic by distributing it to the destinations with various logics (eg. Round robin). I doesn't offer features such as authorisation checks, authentication of requests etc.

Whereas

API Gateway -> Its a managed service provided by various hosting companies to manage API operations to seamlessly scale the API infra. It takes cares of the access control, response caching, response types, authorisation, authentication, request throttling, data handling, identifying the right destinations based on custom rules, and seamless scaling the backend. Generally Managed API gateways by default comes with scalable infra, so putting them behind load balancer might not make sense.

About resolving the Domain, most likely always the DNS resolves to the load balancer, which in turn fetches the response from the API gateway service.

DNS -> Load Balancer -> API gateway -> Backend service

Hope I could explain and clear your confusion.

Philosopher

Just thought I'd add that some load balancers like AWS Application Load Balancer can have authentication rules. – Breast

"putting them behind load balancer might not make sense." but then suggest "DNS -> Load Balancer -> API gateway -> Backend service"? – Outpatient

The same question - why API gateway is placed after the Load Balancer? – Hasdrubal

That's strange ordering, putting Load balancer before api gateway, why? Don't you think we should place Load Balancer after the API gateway? as the request first has to be intercepted before deciding what and where it needs to be redirected. – Prosper

It can be done both before and after depending on the traffic you might have. – Philosopher

Does that mean load balancers balance the load on gateways and not the load on backend services? – Wilburwilburn

@Wilburwilburn You can put loadbalancers before and/or after apiGateways. These are terminologies with their functionalities, how you want to place them depends on the architrecture. If you place before apiGateway, you are effectively balancing the load that goes to apiGateway. After that also, you can have layer 4 load balancers that will load balance your application (UI / backend webserver). – Robespierre

Load balancers (atleast ALBs) can't be put in front of API Gateways, they have to come after. – Thaddeus

Answer 3

There are two scenarios to consider here to clarify the confusion. I have clarified this using microservices example as this would make sense there only.

Scenario 1: You have a cluster of API Gateways

User ---> Load Balancer (provided by Cloud Providers like AWS or your own) ---> API Gateway Cluster ---> Service Discovery Agent (like eureka) ---> Microservice A ---> Client Side Load Balancer ---> Microservice B

Scenario 2: You have a single API Gateway

User ---> API Gateway ---> Service Discovery Agent (like Eureka) ---> Microservice A ---> Client Side Load Balancer -> Microservice B

I hope you understand why we required Load Balancer before the API Gateway in Scenario 1, as there we had multiple instances of API gateway also to handle the large traffic and to avoid the burden on the single api gateway since gateway itself can have several tasks to manage as per the requirements, so to distribute the load among them, we have load balancer.

Inside the microservices environment, you may need load balancing concept at multiple places. Like to accept the outside network you will maintain a load balancer provided by Cloud provider like AWS, eureka (Service Discovery) also acts like a load balancer if there are multiple instances of same service are registered with it and at last we also have client side load balancing (each microservice has its own client side load balancer that maintains a local cache) when your microservices are trying to communicate within them just to avoid burden on service discovery agent(eureka) by not checking every time with it the address for the other microservices.

Note: In the flow diagram, please don't confuse the path from API Gateway --> Service Discovery --> to Microservice as if the Gateway is forwarding request to Service Discovery that later routes it forward. Gateway is just checking for the services registry from the Discovery agents and then routing it to the correct microservice (Not through the Service Discovery agent)

Prosper

scenario 1 well explains the need to load balancer before API Gateway Cluster :) – Bedford

Answer 4

Load Balancer : Its main purpose is to distribute by load balancing traffic between multiple back end systems.

1. We can configure different routes for different back end systems.
2. We get a static ip address for the load balances end points (usually not available with API gateways)
3. Can configure health checks (usually not available with API gateways)

In case of cloud providers, usually "Pay for idle as well"

API Gateway : This as well routes traffic to back end systems based on URL

BUT, its main purpose is targeted towards "API management". Below are such key features which are usually not available in "Load Balancers",

1. Can implement rate limiting, bursting limits
2. Can do request validation and request/response mapping
3. Usually cloud API gateways allows to export/import cross API platform using swagger spec
4. Allows caching responses
   In case of cloud providers, usually "Pay per use"

Gnostic

Answer 5

An API gateway is a reverse proxy that sits between a client and a collection of backend services, allowing the client to access these backend services using a single, centralized ingress point.

As such, an API gateway is most commonly used when an application is implemented using a microservices architecture.

An API gateway accept application programming interface (API) calls for all backend services, conduct the various backend services to process the request (perhaps even breaking one client request into multiple internal requests), combines/aggregates the results, and returns a single response to the client.

The backend services can be deployed on a variety of platforms, such as serverless, cloud-based services, on-premises, etc.

The alternative to an API gateway is to have a load balancer sit in front of each backend service individually. This can make it more complicated for clients to integrate with your API as they may have to make multiple requests, perhaps even using different protocols, to achieve a single goal. By providing an API gateway, your clients can send a single request using a single, secure protocol - the responsibility of coordinating between the different backend services and translating to the correct protocol lies with the API gateway.

Using an API gateway allows you to manage and monitor things that spans across multiple services, such as access control (authentication, authorization), rate limiting/throttling, SSL/TLS termination, analytics, monitoring, billing, managing maintenance schedules, caching response, rules-based routing, etc.

---

A load balancer is a software or hardware device that performs load balancing, with the goal of preventing any one backend/origin server from becoming overloaded.

---

With most cloud providers, you don't need a load balancer in front of the API gateway, as it's the responsibility of the cloud provider to scale the resources backing an API gateway.

An API gateway may route a request to a load balancer instead of directly to a service, where the load balancer would distribute the load evenly amongst multiple instances of the service.

Microampere

Answer 6

DNS is responsible for routing the request to the nearest ip address inside network for a given domain name.

Api gateway is responsible for authentication, finding the correct apis(with or without load balancer) to call and circuit braking, response consolidation.

Load balancer is resposnible for distributing incoming request to different machine having same service deployed on them, on the basis of load or maybe round robin fashion.

So one way of doing it is DNS TO GATEWAY TO LB

NOTE : LB can be placed before gateway depending upon traffic and use case

Circumambient