Why does each microservice get its own database?

Question

It seems that in the traditional microservice architecture, each service gets its own database with a different understanding of the data (described here). Sometimes it is considered permissible for databases to duplicate data. For instance, the "Users" service might know essentially everything about a user, whereas the "Posts" service might just store primary keys and usernames (so that the author of a post can have their name displayed, for instance). This page talks about eventual consistency, sources of truth, and other related concepts when data is duplicated. I understand that microservice architectures sometimes include a shared database, but most places I look suggest that this is a rare strategy.

As for why each service typically gets its own database, all I've seen so far is "so that each service owns its own resources," but I'm not convinced that a) the service layer in any way "owns" the persisted resources accessed through the database to begin with, or that b) services even need to own the resources they require rather than accessing necessary subsets of the master resources through a shared database.

So what are some of the justifications that each service in a microservice architecture should get its own database?

Testate

Answer 1

There are a few reasons why it does make sense to use a separate database per micro-service. Some of them are:

Scaling

Splitting your domain in micro-services is fine. You can scale your particular micro-service on the deployed web-server on demand or scale out as needed. That it obviously one of the benefits when using micro-services. More importantly you can have micro-service-1 running for example on 10 servers as it demands this traffic but micro-service-2 only requires 1 web-server so you deploy it on 1 server. The good thing is that you control this and you can manage your computing resources like in order to save money as Cloud providers are not cheap. Considering this what about the database? If you have one database for multiple services you could not do this. You could not scale the databases individually as they would be on one server.

Data partitioning to reduce size

Automatically as you split your domain in micro-services with each containing 1 database you split the amount of data that is stored in each database. Ideally if you do this you can have smaller database servers with less computing power and/or RAM. In general paying for multiple small servers is cheaper then one large one. So in this case you could make use of this fact and save some resources as well. If it happens that the already spited by domain database have large amount of data techniques like data sharding or data partitioning could be applied additional, but this is another topic.

Which db technology fits the business requirement

This is very important pro fact for having multiple databases. It would allow you to pick the database technology which fits your Business requirement best in order to get the best performance or usage of it. For example some specific micro-service might have some Read-heavy operations with very complex filter options and a full text search requirement. Using Elastic Search in this case would be a good choice. Some other micro-service might use SQL Server as it requires SQL specific features like transnational behavior or similar. If for some reason you have one database for all services you would be stuck with the particular database technology which might not be so performant for those requirement. It is a compromise for sure.

Developer discipline

If for some reason you would have a couple micro-services which would share their database you would need to deal with the human factor. The developers would need to be disciplined to not cross domains and access/modify the other micro-services database(tables, collections and etc) which would be hard to achieve and control. In large organisations with a lot of developers this could be a serious problem. With a hard/physical split this is not an issue.

Summary

There are some arguments for having database per micro-service but also some against it. In general the guidelines and suggestions when using micro-services are to have the micro-service together with its data autonomous in order to work independent in Ideal case(this is not the case always). It is defiantly a compromise as well as using micro-services in general. As always the rule is the rule but there are exceptions to it. Micro-services architecture is flexible and very dependent of your Domain needs and requirements. If you and your team identify that it makes sense to merge multiple micro-service databases to 1 and that it solves a lot of your problems then go for it.

Enfranchise

Well, since we're talking about scalability, horizontal sharding of data is a must. There's effectively no difference between storing all data in one schema and sharding it rather than sharding several schemas, other than the shard index selection to avoid cross-sharded queries. Database-per-service may help for this in some way, but not the optimal way, which is to just have a very small dedicated database for each shard index. As such, one can have perfectly scalable databases and still have far fewer schemas than services. I do agree with your third point, and somewhat with your fourth. – Testate

Sure I agree with your points, but nobody said that micro-services must have database per service. If for your case it seems to be better to merge a couple in 1 db go for it ;) . By the way in one of my previous project we where doing exactly that. We had to go from monolithic architecture to micro-services and somewhere in between we ended up having hybrid Architecture with some services sharing one database. Sure it was a transition phase but it worked as we had it deployed on Production and it was working just fine. I am just trying to give you valid points why this could make sense. – Enfranchise

Answer 2

I am really unconvinced by this argument that your app can carry on even if one microservice fails. Yes perhaps your app can run and continue to let customers browse your product catalog, but how much use is that really if you cannot take orders, which involves first checking inventory and you cannot take payment. Far more likely that if the customer finds a product they like and then cannot buy it from you, they will find that product or a similar product elsewhere. Technically part of your app is still running but in most cases it is useless unless the whole app is running. Therefore this is not a reason against a single shared database vs one database per micro service.

Everyplace

I agree, but more importantly, a more appropriate way to support failover is by mirroring behind a gateway. That's true for databases, web servers, etc, regardless of whether they're implemented as monoliths or microservices. Microservices have virtually nothing to do with failover. They just accidentally provide a poor form of failover that, as you say, isn't sufficient for any real application anyway. – Testate

Answer 3

Microservices

Microservices advocate design constraints where each service is developed, deployed and scaled independently. This philosophy is only possible if you have database per service. How can i continue my business if i have DB failure and what steps i can take to mitigate this?DB is essential part of any enterprise application. I agree there are different number of challenges when services has its own databases.

Why Independent database?

Unlike other approaches this approach not only keeps your code-base clean and extendable but you truly omit the single point of failure in your business. To achieve this services sometimes can have duplicated data as well, as long as my service is autonomous and services can only be autonomous if i have database per service.

From business point of view, Lets take eCommerce application. you have microserivces like Booking, Order, Payment, Recommendation , search and so on. Database is shared. What happens if the DB is down ? All your services are down ! and there is no point using Microservies architecture other than you have clean code base.

If you have each service having it's own database , i don't mind if my recommendation service is not working but i can still search and book the order and i haven't lost the customer. that's the whole point.

It comes at cost and challenges, but in longer run it pays off.

SQL / NoSQL

Each service has it's own needs. To get the best performance I can use SQL for payment service (transaction) and I can use (I should) NoSQL for recommendation service. Shared database wouldn't help me in this case. In modern cloud Architectures like CQRS, Event Sourcing, Materialized views, we sometimes use 2 different databases for same service to get the performance out of it.

Again Database per service is not only about resources or how much data should it own. But we really have to see the bigger picture. Yes we have certain practices how much data and duplication is good or bad but that's another debate.

Hope that helps !

Operand

"What happens if the DB is down ? All your services are down !" Database-per-service does not inherently have anything to do with failover; only database-server-per-service does this. If the separate databases are hosted on the same machine (not uncommon), there's still a single point of failure. See here: microservices.io/patterns/data/database-per-service.html. If you want to protect your entire application from failover (the smarter approach), use mirroring: docs.microsoft.com/en-us/sql/database-engine/database-mirroring/… – Testate

As for "Each service has it's own needs," I completely agree, but when many of the services' needs overlap, this is not a reason to give each service its own database. – Testate

Good Point, however fail-over that impact one service at a time vs impact multiple services at a time due to shared DB is a business call. Business critical DBs are separate physically as well. May be if business services are non-trivial and i can take shared db approach but that would an anti pattern. Just like Other Software patterns SOLID, DRY etc (which not everyone follows) i can choose not to but at the cost of scale and impact. Imagine you have to do 10 release a day across 10 micro-services, how can we share the database without risking impact other teams? – Operand

These are good practices like any other software design principles and pattern but i can choose not to. The benefits like scale-ability , availability , less impact area and having autonomous services are far more practical and important than having a shared DB. Most the enterprises choose go that path which comes at cost and challenge but benefits are long lasting. Again i think in the end it's a pattern but you will have to choose based on your needs. It will all come down to your RTO and RPO. Cheers ! – Operand

That's actually a very helpful approach to looking at this problem. I've never considered it similar to a software pattern. I always thought it was just typically considered a hard and fast rule of microservices. Thanks! – Testate

That's right. If we avoid choosing this pattern, may be we are not building true microservices or at least not following recommended practices around it. – Operand