AccessControlException when .Net Client App accessing Azure Data Lake

Question

I am trying to access a Data Lake from a .Net client application using this example

I have registered the Client App in AAD Tenant and using the Client Id and Client secret from there (as what i believe is service-to-service authentication.)

The Data Lake is in a different subscription but belongs to the same Tenant/AAD

The App has Read/Write/Execute permission under 'Owner' and 'Assigned Permissions' for the specific folder (two hierarchies down the root folder) in the datalake. The parent folders upto the root have Execute permissions as mentioned here. The overall level access in 'Access Control (IAM)' for the app is 'Reader'

I get the following error which I believe means I am able to authenticate but do not have enough permissions to read the read/write:

Microsoft.Azure.DataLake.Store.AdlsException: Error opening a Read Stream for file something/something/something.txt
Operation: GETFILESTATUS failed with HttpStatus:Forbidden RemoteException: AccessControlException GETFILESTATUS failed with 
error 0x83090aa2 (Forbidden. ACL verification failed. Either the resource does not exist or the user is not authorized to 
perform the requested operation.).
[***][***] JavaClassName: org.apache.hadoop.security.AccessControlException.
Last encountered exception thrown after 1 tries. [Forbidden: AccessControlException]
[ServerRequestId:***]

I fail to understand what other permissions are missing? Do I have to use service principals here? If so how do I check what is the acess for my App's service principal on this data lake. Thanks.

Baby

Answer 1

The answer is - Yes you have to user Service Principals.

Please see here for more information about Applications vs Service Principals

Basically Azure uses Service Principle ID in the background to authorize data access in Data Lake. If you use the Application ID, you will see the same 'Display Name' for you application on the Data Lake Folder's 'Access' blade, and wrongly think you have access.

So make sure to use Service Principal ID and not the Application ID to grant ACL access.

Baby

Answer 2

easiest way to test, grant your app owner permissions on the data lake (Access Control IAM), wait 15 minutes and test. if that fixes it >> this means you messed up on permissions somewhere.

Impossible to tell where you did an error in your permissions assignment. as a general rule, you need read\execute to read files (which is a bit of a surprise).

Cyprinoid

But when I drill down to the folder that I am using, my app has owner permissions for that folder with r/w/x. Unfortunately data lake is owned by somebody else (who has no clue how to fix this) and therefore I do not have admin rights to change permissions. – Baby

well, the error is rights related, you claim you assigned them properly but nothing is working, so it means you think you've assigned them properly (did you wait 15-30 minutes before testing?). another thing you might want to do is assign r\w\x at the root and propagate that to all the folders and files. but assigning owner is a lot easier and less error prone – Cyprinoid