Using KeyCloak(OpenID Connect) with Apache SuperSet

Question

I started with Using OpenID/Keycloak with Superset and did everything as explained. However, it is an old post, and not everything worked. I'm also trying to implement a custom security manager by installing it as a FAB add-on, so as to implement it in my application without having to edit the existing superset code.

I'm running KeyCloak 4.8.1.Final and Apache SuperSet v 0.28.1

As explained in the post, SuperSet does not play nicely with KeyCloak out of the box because it uses OpenID 2.0 and not OpenID Connect, which is what KeyCloak provides.

The first difference is that after pull request 4565 was merged, you can no longer do:

from flask_appbuilder.security.sqla.manager import SecurityManager

Instead, you now have to use: (as per the UPDATING.md file)

from superset.security import SupersetSecurityManager

In the above mentioned post, the poster shows how to create the manager and view files separately, but don't say where to put it. I placed both the manager and view classes in the same file, named manager.py, and placed it in the FAB add-on structure.

from flask_appbuilder.security.manager import AUTH_OID
from superset.security import SupersetSecurityManager
from flask_oidc import OpenIDConnect
from flask_appbuilder.security.views import AuthOIDView
from flask_login import login_user
from urllib.parse import quote
from flask_appbuilder.views import ModelView, SimpleFormView, expose
import logging

class OIDCSecurityManager(SupersetSecurityManager):
    def __init__(self,appbuilder):
        super(OIDCSecurityManager, self).__init__(appbuilder)
        if self.auth_type == AUTH_OID:
            self.oid = OpenIDConnect(self.appbuilder.get_app)
        self.authoidview = AuthOIDCView

CUSTOM_SECURITY_MANAGER = OIDCSecurityManager

class AuthOIDCView(AuthOIDView):
    @expose('/login/', methods=['GET', 'POST'])
    def login(self, flag=True):
        sm = self.appbuilder.sm
        oidc = sm.oid

        @self.appbuilder.sm.oid.require_login
        def handle_login(): 
            user = sm.auth_user_oid(oidc.user_getfield('email'))

            if user is None:
                info = oidc.user_getinfo(['preferred_username', 'given_name', 'family_name', 'email'])
                user = sm.add_user(info.get('preferred_username'), info.get('given_name'), info.get('family_name'), info.get('email'), sm.find_role('Gamma')) 

            login_user(user, remember=False)
            return redirect(self.appbuilder.get_url_for_index)  

        return handle_login()  

@expose('/logout/', methods=['GET', 'POST'])
def logout(self):
    oidc = self.appbuilder.sm.oid
    oidc.logout()
    super(AuthOIDCView, self).logout()        
    redirect_url = request.url_root.strip('/') + self.appbuilder.get_url_for_login
    return redirect(oidc.client_secrets.get('issuer') + '/protocol/openid-connect/logout?redirect_uri=' + quote(redirect_url))

I have the CUSTOM_SECURITY_MANAGER variable set in this file and not in superset_config.py. This is because it didn't work when it was there, it didn't load the custom security manager. I moved the variable there after reading Decorator for SecurityManager in flask appbuilder for superest.

My client_secret.json file looks as follows:

{
    "web": {
        "realm_public_key": "<PUBLIC_KEY>",
        "issuer": "https://<DOMAIN>/auth/realms/demo",
        "auth_uri": "https://<DOMAIN>/auth/realms/demo/protocol/openid-connect/auth",
        "client_id": "local",
        "client_secret": "<CLIENT_SECRET>",
        "redirect_urls": [
            "http://localhost:8001/*"
        ],
        "userinfo_uri": "https://<DOMAIN>/auth/realms/demo/protocol/openid-connect/userinfo",
        "token_uri": "https://<DOMAIN>/auth/realms/demo/protocol/openid-connect/token",
        "token_introspection_uri": "https://<DOMAIN>/auth/realms/demo/protocol/openid-connect/token/introspect"
    }
}

• realm_public_key: I got this key at Realm Settings > Keys > Active and then in the table, in the "RS256" row.
• client_id: local (the client I use for local testing)
• client_secret: I got this at Clients > local (from the table) > Credentials > Secret

All the url/uri values are adjusted from the first mentioned post I used to set it all up. The <DOMAIN> is an AWS CloudFront default domain, since I'm running KeyCloak on EC2 and don't want to go through the trouble to setup a custom HTTPS domain for simply getting it up and running.

Then, finally, part of my superset_config.py file looks like this:

ADDON_MANAGERS = ['fab_addon_keycloak.manager.OIDCSecurityManager']
AUTH_TYPE = AUTH_OID
OIDC_CLIENT_SECRETS = '/usr/local/lib/python3.6/site-packages/fab_addon_keycloak/fab_addon_keycloak/client_secret.json'
OIDC_ID_TOKEN_COOKIE_SECURE = False
OIDC_REQUIRE_VERIFIED_EMAIL = False
AUTH_USER_REGISTRATION = True
AUTH_USER_REGISTRATION_ROLE = 'Gamma'
OPENID_PROVIDERS = [{
    'name': 'KeyCloak',
    'url': 'https://<DOMAIN>/auth/realms/demo/account'
}]

In the original post, the OPENID_PROVIDERS environment variable is not mentioned, so I'm not really sure what to put in here for the URL. I put that one since that's the URL you'll hit to login to the client console on KeyCloak.

When I run SuperSet I don't get any errors. I can see that the custom security manager loads. When I navigate to the login screen, I have to choose my provider, I don't get a login form. I choose KeyCloak, since there's obviously nothing else, and click Login. When I click Login I can see that something loads in the address bar of the browser, but nothing happens. It's my understanding that I'm supposed to be redirected to the KeyCloak login form, and then back to my application upon successful login, but nothing happens. Am I missing something somewhere?

Edit

So after some more digging, it seems like my custom view class loads, however the methods in the class do not override the default behavior. Not sure why this is happening or how to fix it.

Revision

Answer 1

I ended up figuring it out myself.

The solution I ended up with does not make use of a FAB add-on, but you also don't have to edit existing code/files.

I've renamed the manager.py file to security.py, and it now looks like this:

from flask import redirect, request
from flask_appbuilder.security.manager import AUTH_OID
from superset.security import SupersetSecurityManager
from flask_oidc import OpenIDConnect
from flask_appbuilder.security.views import AuthOIDView
from flask_login import login_user
from urllib.parse import quote
from flask_appbuilder.views import ModelView, SimpleFormView, expose
import logging

class AuthOIDCView(AuthOIDView):

    @expose('/login/', methods=['GET', 'POST'])
    def login(self, flag=True):
        sm = self.appbuilder.sm
        oidc = sm.oid

        @self.appbuilder.sm.oid.require_login
        def handle_login(): 
            user = sm.auth_user_oid(oidc.user_getfield('email'))

            if user is None:
                info = oidc.user_getinfo(['preferred_username', 'given_name', 'family_name', 'email'])
                user = sm.add_user(info.get('preferred_username'), info.get('given_name'), info.get('family_name'), info.get('email'), sm.find_role('Gamma')) 

            login_user(user, remember=False)
            return redirect(self.appbuilder.get_url_for_index)  

        return handle_login()  

    @expose('/logout/', methods=['GET', 'POST'])
    def logout(self):

        oidc = self.appbuilder.sm.oid

        oidc.logout()
        super(AuthOIDCView, self).logout()        
        redirect_url = request.url_root.strip('/') + self.appbuilder.get_url_for_login

        return redirect(oidc.client_secrets.get('issuer') + '/protocol/openid-connect/logout?redirect_uri=' + quote(redirect_url))

class OIDCSecurityManager(SupersetSecurityManager):
    authoidview = AuthOIDCView
    def __init__(self,appbuilder):
        super(OIDCSecurityManager, self).__init__(appbuilder)
        if self.auth_type == AUTH_OID:
            self.oid = OpenIDConnect(self.appbuilder.get_app)

I place the security.py file next to my superset_config_py file.

The JSON configuration file stays unchanged.

Then I've changed the superset_config.py file to include the following lines:

from security import OIDCSecurityManager
AUTH_TYPE = AUTH_OID
OIDC_CLIENT_SECRETS = <path_to_configuration_file>
OIDC_ID_TOKEN_COOKIE_SECURE = False
OIDC_REQUIRE_VERIFIED_EMAIL = False
AUTH_USER_REGISTRATION = True
AUTH_USER_REGISTRATION_ROLE = 'Gamma'
CUSTOM_SECURITY_MANAGER = OIDCSecurityManager

That's it.

Now when I navigate to my site, it automatically goes to the KeyCloak login screen, and upon successful sign in I am redirected back to my application.

Revision

There already exists a file with a name security.py in the superset folder. Did you edited the same file ? – Fenella

Hi @DeepaMG, no I did not edit that file. I have a folder in my home directory "/superset/" that has my superset_config.py file. I placed the security.py and JSON files next to the superset_config.py file. If your superset_config.py file is in the superset package directory which already has the security.py file, you can call it something else (e.g. oidc_security.py) and just change the reference in your superset_config.py file accordingly. – Revision

Yes, I got it working. @sj-meyer What i am getting now is 1. On hitting URL of superset, it redirects me to keycloak. 2. After entering the credentials, i am able to see superset. The thing is i need to have a same user credentials in keycloak as well as in superset. Cant i just have credentials in keycloak only ? – Fenella

I'm not sure, I don't think so. I've set mine up so that a user is created automatically if they login to KeyCloak but don't exist on superset. What also happens when creating a new user on KeyCloak, it will check superset and if there is a user with the same email address (and I think username) it will log you into that existing user. The fact that users are created in superset is a good thing in my opinion. It allows you to leverage the built-in roles of superset to manage permissions. – Revision

But i am iframing the superset into my another application which is already integrated with keycloak which have lacks of user. I do not want to create a same credentials again in superset. – Fenella

Ok, so you have an existing application with users, and now you want those users to have access to superset, and you want to use KeyCloak to manage auth for both of these applications? – Revision

yes. Is there are way to do that with changes in configurations? – Fenella

How did you manage to create user is automatically if they login to KeyCloak but don't exist on superset? Can you add that to answer? – Fenella

Also superset still remain logged in on refresh even if i clear a login sessions from a keycloak admin. gets logged out only after i click log-out button. Isnt this a serious issue !! – Fenella

@DeepaMGWhat you can do is to configure KeyCloak to use your existing users database as it's own. That way KeyCloak will hit the existing database of users when logging into Superset and there will be no need for replication of users. As for setting up Superset to create a user if it exists on KeyCloak but not in Superset; my answer does show how to do that. AUTH_USER_REGISTRATION = True AUTH_USER_REGISTRATION_ROLE = 'Gamma' Lastly, I haven't fully tested the behavior, so I don't know the answer to that question. – Revision

What I'm thinking of doing to prevent the scenario you just described is to set the Superset cookie timeout to be really low, like 5 minutes or so. That way if you destroy the session on KeyCloak, it'll take a max of 5 minutes for a user to be kicked out of Superset. This theory is based on some assumptions I'm making, I haven't tested it yet. – Revision

First thanks, i managed to have Keyloak login working if user is None: info = oidc.user_getinfo(['preferred_username', 'given_name', 'family_name', 'email']) user = sm.add_user(info.get('preferred_username'), info.get('given_name'), info.get('family_name'), info.get('email'), sm.find_role('Gamma')) This code adds the user to superset's database on first login. Is the a possibility tuo update the user with the roles from keycloak on each login? I managed to extract the roles from the token but couldn't ind the right syntax to update the user. – Graptolite