Error 403 User not authorized when trying to access Azure Databricks API through Active Directory

Asked 1/9, 2020 at 10:45 Answered 13/5 at 15:16

Solved azure oauth-2.0 azure-active-directory azure-databricks

I have been following the document: https://learn.microsoft.com/en-us/azure/databricks/dev-tools/api/latest/aad/service-prin-aad-token

to create a service principal and use it to access Databricks. I already have a databricks workspace configured and have used it to create a cluster. Then I've followed the process mentioned in the document, created a service principal and obtained the two tokens: AD Access token and management access token. However, I am unable to use the API.

The final call cURL command after configuration:

curl -X GET \
-H 'Authorization: Bearer <access-token>' \
-H 'X-Databricks-Azure-SP-Management-Token: <management-access-token>' \
-H 'X-Databricks-Azure-Workspace-Resource-Id: /subscriptions/<subscription-id>/resourceGroups/<resource-group-name>/providers/Microsoft.Databricks/workspaces/<workspace-name>' \
https://<databricks-instance>/api/2.0/clusters/list

With the tokens and the other info substitutued provides the following result:

<html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=utf-8"/>
<title>Error 403 User not authorized.</title>
</head>
<body><h2>HTTP ERROR 403</h2>
<p>Problem accessing /api/2.0/clusters/list. Reason:
<pre>    User not authorized.</pre></p>
</body>
</html>

I have tried a couple of things to resolve this, namely changing the original access token request to use the interactive flow using the authorization_code grant type and so on, but that just gives me an Invalid access token error.

Is there something wrong with the above configuration? Am I missing some permissions?

Update: Came to the Access control (IAM) page and the app does not seem to be included there.

Maine answered 1/9, 2020 at 10:45 Comment(5)

Are you either a Contributor or Owner role on the Databricks workspace resource in Azure ? – Gallivant 1/9, 2020 at 12:49

The service principal must be added to the workspace either as part of the admin user login or using the Add service principal endpoint. – Mesial 2/9, 2020 at 7:37

@Axel I am a contributor but despite that I was not able to provide access. Have requested access from my admin separately. – Maine 2/9, 2020 at 14:58

@CarlZhao Will obtain access and try this out. – Maine 2/9, 2020 at 14:59

If my answer is helpful for you, you can accept it as answer( click on the check mark beside the answer to toggle it from greyed out to filled in.). See meta.stackexchange.com/questions/5234/… This can be beneficial to other community members. Thank you. – Mesial 3/9, 2020 at 5:41

You must grant a role to the service principal.

This is the result of my test that I did not grant a role to the service principal. The error is the same as yours:

Next, grant roles to the service principal according to the following process:

Azure Portal>Azure Databricks>Azure Databricks Service>Access control (IAM)>Add a role assignment>Select the role you want to grant and find your service principal>save

Finally, use the service principal to get the token.(Don’t forget to grant permissions to service principals and grant administrator consent)

Get an Azure Active Directory access token: Get the Azure Management Resource endpoint token: Use the management endpoint access token to access the Databricks REST API:

Mesial answered 3/9, 2020 at 4:9 Comment(0)

I also faced the same error and issue persisted in SPN configuration in databricks. Basically, wrong client id was mention while granting access to SPN in databricks.

How to verify?

In databricks: Go to settings > Identity and Access > Service principals > Then verify 'Application Id' is matching in Azure portal

In azure portal: Search for SPN and go it Application (not SPN). Verify 'Application (client) ID' is matching with databricks from previous step.

Once I sync up this, I was able to use SPN to make successful API call.

Koal answered 13/5 at 15:16 Comment(0)

In case you wish to access the Databricks endpoints with just the access token, as is the case with using DBX in CI/CD workflows to trigger the Databricks pipelines, you would need to add the service principal as a user in the Databricks workspace. So then only the access token would be needed.

Refer to the documentation here- Add Service Principal as Databricks user

Aveyron answered 18/5, 2023 at 13:19 Comment(0)

Recommended topics

Hot tags