Multi-tenancy: Managing multiple datasources with Spring Data JPA
Asked Answered
R

1

19

I need to create a service that can manage multiple datasources. These datasources do not necessarily exist when the app when first running the app, actually an endpoint will create new databases, and I would like to be able to switch to them and create data.

For example, let's say that I have 3 databases, A, B and C, then I start the app, I use the endpoint that creates D, then I want to use D.

Is that possible?

I know how to switch to other datasources if those exist, but I can't see any solutions for now that would make my request possible. Have you got any ideas?

Thanks

Representationalism answered 10/4, 2018 at 17:26 Comment(4)
In that case I would abandon using datasources as beans and JPA repositories. Spring Beans are fine if they are static- created at app startup. You could create datasources and EntityManager programatically (using factory methods).Eade
Please look at my demo project for now. A little later I will prepare an answer with a description...Fidelfidela
Please don't forget to accept/upvote answers that helped you...Fidelfidela
Sorry about that, thank you very much for your help and your whole answer. I really appreciate it and it works very well!Representationalism
F
43

To implement multi-tenancy with Spring Boot we can use AbstractRoutingDataSource as base DataSource class for all 'tenant databases'.

It has one abstract method determineCurrentLookupKey that we have to override. It tells the AbstractRoutingDataSource which of the tenant datasource it has to provide at the moment to work with. Because it works in the multi-threading environment, the information of the chosen tenant should be stored in ThreadLocal variable.

The AbstractRoutingDataSource stores the info of the tenant datasources in its private Map<Object, Object> targetDataSources. The key of this map is a tenant identifier (for example the String type) and the value - the tenant datasource. To put our tenant datasources to this map we have to use its setter setTargetDataSources.

The AbstractRoutingDataSource will not work without 'default' datasource which we have to set with method setDefaultTargetDataSource(Object defaultTargetDataSource).

After we set the tenant datasources and the default one, we have to invoke method afterPropertiesSet() to tell the AbstractRoutingDataSource to update its state.

So our 'MultiTenantManager' class can be like this:

@Configuration
public class MultiTenantManager {

    private final ThreadLocal<String> currentTenant = new ThreadLocal<>();
    private final Map<Object, Object> tenantDataSources = new ConcurrentHashMap<>();
    private final DataSourceProperties properties;

    private AbstractRoutingDataSource multiTenantDataSource;

    public MultiTenantManager(DataSourceProperties properties) {
        this.properties = properties;
    }

    @Bean
    public DataSource dataSource() {
        multiTenantDataSource = new AbstractRoutingDataSource() {
            @Override
            protected Object determineCurrentLookupKey() {
                return currentTenant.get();
            }
        };
        multiTenantDataSource.setTargetDataSources(tenantDataSources);
        multiTenantDataSource.setDefaultTargetDataSource(defaultDataSource());
        multiTenantDataSource.afterPropertiesSet();
        return multiTenantDataSource;
    }

    public void addTenant(String tenantId, String url, String username, String password) throws SQLException {

        DataSource dataSource = DataSourceBuilder.create()
                .driverClassName(properties.getDriverClassName())
                .url(url)
                .username(username)
                .password(password)
                .build();

        // Check that new connection is 'live'. If not - throw exception
        try(Connection c = dataSource.getConnection()) {
            tenantDataSources.put(tenantId, dataSource);
            multiTenantDataSource.afterPropertiesSet();
        }
    }

    public void setCurrentTenant(String tenantId) {
        currentTenant.set(tenantId);
    }

    private DriverManagerDataSource defaultDataSource() {
        DriverManagerDataSource defaultDataSource = new DriverManagerDataSource();
        defaultDataSource.setDriverClassName("org.h2.Driver");
        defaultDataSource.setUrl("jdbc:h2:mem:default");
        defaultDataSource.setUsername("default");
        defaultDataSource.setPassword("default");
        return defaultDataSource;
    }
}

Brief explanation:

  • map tenantDataSources it's our local tenant datasource storage which we put to the setTargetDataSources setter;

  • DataSourceProperties properties is used to get Database Driver Class name of tenant database from the spring.datasource.driverClassName of the 'application.properties' (for example, org.postgresql.Driver);

  • method addTenant is used to add a new tenant and its datasource to our local tenant datasource storage. We can do this on the fly - thanks to the method afterPropertiesSet();

  • method setCurrentTenant(String tenantId) is used to 'switch' onto datasource of the given tenant. We can use this method, for example, in the REST controller when handling a request to work with database. The request should contain the 'tenantId', for example in the X-TenantId header, that we can retrieve and put to this method;

  • defaultDataSource() is build with in-memory H2 Database to avoid using the default database on the working SQL server.

Note: you must set spring.jpa.hibernate.ddl-auto parameter to none to disable the Hibernate make changes in the database schema. You have to create a schema of tenant databases beforehand.

A full example of this class and more you can find in my repo.

UPDATED

This branch demonstrates an example of using the dedicated database to store tenant DB properties instead of property files (see the question of @MarcoGustavo below).

Fidelfidela answered 11/4, 2018 at 20:19 Comment(15)
I was able to make it work, but what i want to is to load dataSource from a database instead .properties file. Is it possible to create a tenantResolver retrieving these data from a repository that uses another dataSource?Oblivious
@MarcoGustavo Why not? Instead of property files you can create a table, for example, in your default database, to store tenant database props. Then create a repo for it and maybe a service and use them in tenantResolver method...Fidelfidela
I created a class called TenantResolverFactory with my repo and im using it this way: TenantResolverFactory tenantResolverFactory = new TenantResolverFactory(); this.tenantManager.setTenantResolver(tenantResolverFactory::tenantResolver); on the Application constructor, but my repository is null in my TenantResolverFactory class. Could you please provide me an example?Oblivious
@MarcoGustavo Check tenant-in-db branch of my example. The implementation turned out to be a bit complicated than it seemed at first glance - I had to create two datasources: one for AbstractRoutingDataSource, the second - for the database where the tenant props are stored (in this style). To run the branch - firts start Postgres with my docker-compose.yml - it creates predefined tenant DBs, then run Application. Then you can use demo requests in api-demo.http..Fidelfidela
Much better than i thought. Works like a charm. Thank you!Oblivious
Thanks for the elaborated explaination, I was looking to achieve multitenancy using spring JPA and not the hibernate classes. Brilliantly explained.Ingles
Sorry for bumping after 2 years, but I checked your code and the Branch where the tenants are saved in the DB, I am trying to implement it, but at the moment when I change the tenant in the MultiTenantManager the respective database is not changed. Can you help me @Fidelfidela ? Thank you.Berrie
@Berrie Just checkout tenant-in-db branch, run command docker-compose up -d, then run sample requests from api-demo.http in IDEA...Fidelfidela
@Fidelfidela I mean I am implementing your solution in my project. I am using the app users as tenants and the logic is that whenever a user signs up, a DB will be created for them and in the "AdminDB" where the users and Tenants are located, a new record in the Tenants table will be created. then when the user signs in (I use JWT) the tenant in the ThreadLocal will be set to him. But when I do that, the database connection won't change to him. Everything else works, just changing the DB from AdminDB to TenantN doesn't work. Do you have any idea what could be the problem? Thank you for everything :)Berrie
@Berrie I'm afraid I can't help you with your app. Try to compare your implementation with mine then find the issue..Fidelfidela
@Fidelfidela I'll try to find it and write a comment with the solution, maybe someone else bumps into the same problem, Thank you for your time and solution.Berrie
@Fidelfidela and anyone else interested, I found my issue, I was setting the currentTenant on the wrong filter of Spring Security. That was my Issue. If I can ask you one last question, do you know the best way to create the database through Spring programmatically every time the user signs up ? Thanks so so much, your solution helped me tremendously.Berrie
@Berrie "the best way to create the database through Spring programmatically every time the user signs up" - In my prev work we did such with Amazon Lambda and poor JDBC (our DBs were hosted on Amazon RDS). If I remember correctly, we published the corresponding event in the SQS with new client parameters, the Lamba was subscribed to this event and created a database for it.Fidelfidela
Hi, @Fidelfidela Thank you for the post on multi-tenancy. I have implemented like shown above. I wonder how to make a balance between performance and memory with this approach. For example, let us say I have opened 20 connections and added them in my hashmap against a key. Now, How can I clear up the idle connections and also remove the corresponding entry from the hashmap because we do not want our connection pool and hashmap to grow indefinitely. link please suggestPhotoplay
DataSource dataSource = DataSourceBuilder.create() .driverClassName(properties.getDriverClassName()) .url(url) .username(username) .password(password) .build(); This line is always giving me null, not able to resolveIndigestion

© 2022 - 2024 — McMap. All rights reserved.