EXTERNAL TABLE access failed due to internal error: 'Java exception raised on call to HdfsBridge_IsDirExist. Java exception message:

Question

I am trying to create external table through polybase with below syntax on Visual Studio 2015. Its giving me the below error. Can some one pls help on this

CREATE EXTERNAL TABLE dbo.DimDate2External (
    DateId INT NOT NULL,
    CalendarQuarter TINYINT NOT NULL,
    FiscalQuarter TINYINT NOT NULL
)
WITH (
    LOCATION='/textfiles/DimDate2.txt',
    DATA_SOURCE=AzureStorage,
    FILE_FORMAT=TextFile
);

CREATE EXTERNAL DATA SOURCE AzureStorage
WITH ( 
    TYPE = HADOOP, 
    LOCATION = 'wasbs://<blob_container_name>@<azure_storage_account_name>.‌​blob.core.windows.ne‌​t', 
    CREDENTIAL = AzureStorageCredential
    ); 

CREATE EXTERNAL FILE FORMAT TextFile WITH ( FORMAT_TYPE = DelimitedText, FORMAT_OPTIONS (FIELD_TERMINATOR = ',') );

EXTERNAL TABLE access failed due to internal error:

'Java exception raised on call to HdfsBridge_IsDirExist. Java exception message: com.microsoft.azure.storage.StorageException: Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.: Error [com.microsoft.azure.storage.StorageException: Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.] occurred while accessing external file.'

Cutinize

Can you post the definition for your external data source and file format please? Remove any IP addresses, information etc that is necessary. – Semblance

CREATE EXTERNAL DATA SOURCE AzureStorage WITH ( TYPE = HADOOP, LOCATION = 'wasbs://<blob_container_name>@<azure_storage_account_name>.blob.core.windows.net', CREDENTIAL = AzureStorageCredential ); CREATE EXTERNAL FILE FORMAT TextFile WITH ( FORMAT_TYPE = DelimitedText, FORMAT_OPTIONS (FIELD_TERMINATOR = ',') ); – Cutinize

Just to rule out the obvious, the Azure Blob Storage account has a container called your container with a folder called 'textfiles' and a file called DimDate2.txt in it right? – Semblance

There are three areas to debug this in SQL DW. 1) The Database scoped credential (make sure that the key is correct), 2) the External Data Source (make sure that the container and URI are correct. Case does matter) 3) The External Table definition (Make sure that the location is specified correctly, again case matters.) – Sorrento

Answer 1

In the 'LOCATION' syntax I mistakenly misplaced the Blob container and Storage account and got this error. Now its fixed.

CREATE EXTERNAL DATA SOURCE AzureStorage WITH ( TYPE = HADOOP, LOCATION = 'wasbs://@.‌​blob.core.windows.ne‌​t', CREDENTIAL = AzureStorageCredential )

Cutinize

Answer 2

I can reproduce this error if the Azure Storage account element of your external data source is incorrect (XXX in my example):

CREATE EXTERNAL DATA SOURCE eds_dummy
WITH (  
    TYPE = Hadoop,
    LOCATION = 'wasbs://[email protected]',
    CREDENTIAL = sc_tpch
);

If the blob container name is incorrect (dummy in my example) but the storage account is correct, you get a very specific error message when trying to create the table:

Msg 105002, Level 16, State 1, Line 27 EXTERNAL TABLE access failed because the specified path name '/test.txt' does not exist. Enter a valid path and try again.

There appears to be some kind of validation on the blob container. However if the Azure Storage Account name is incorrect, you do not get an error when you create the external data source, only when you try and create the table:

Msg 105019, Level 16, State 1, Line 35 EXTERNAL TABLE access failed due to internal error: 'Java exception raised on call to HdfsBridge_IsDirExist. Java exception message: com.microsoft.azure.storage.StorageException: The server encountered an unknown failure: : Error [com.microsoft.azure.storage.StorageException: The server encountered an unknown failure: ] occurred while accessing external file.'

To correct, please make sure the Azure Storage Account and Blob container exist.

The easiest way to do this is copy the URL of your file or folder from the portal and fix it up for external tables, ie from this:

https://yourStorageAccountName.blob.core.windows.net/yourBlobContainerName

to this:

wasbs://[e-mail]

Good luck.

Semblance

Thanks. In the 'LOCATION' syntax I mistakenly misplaced the Blob container and Storage account and got this error. Now its fixed. CREATE EXTERNAL DATA SOURCE AzureStorage WITH ( TYPE = HADOOP, LOCATION = 'wasbs://<blob_container_name>@<azure_storage_account_name>.blob.core.windows.net', CREDENTIAL = AzureStorageCredential ); – Cutinize