How can I connect Databricks Community Edition cluster from PyCharm

Question

I want to work on some small exercise projects, I wish to use databricks cluster. Can this be done. I am hoping there is some way to connect databricks cluster through databricks-connect utility. Just need some steps. Thanks in advance.

Bang

Answer 1

You might have already known this by now but adding this for new users.

Community Edition doesn't support databricks-connect functionality. It's only available for the enterprise version. This is the reason there is a no token generation available for community edition.

Goethite

Answer 2

The feature you aer looking for is called "databricks connect". You can configure several EDIs (e.g. Pycharm) to connect live to your Cluster on databricks community as well as Azure and AWS.

If you can understand German by any chance, this youtube video might help you. (Shows the full installation process for windows 10).

https://www.youtube.com/watch?v=VTfRh_RFzOs&t=3s

Smaragdine

Thanks Benjamin but I am asking for connecting to Databricks Community Edition Cluster – Bang

Yeah that is basically the same. Only the part with the ports differs. – Smaragdine

Where do I get a User Token for Community edition of Databricks? – Hirai

- Click the user profile icon User Profile in the upper right corner of your Databricks workspace. - Click User Settings. - Go to the Access Tokens tab. - Click the Generate New Token button. - Optionally enter a description (comment) and expiration period. - Click the Generate button. - Copy the generated token and store in a secure location. Link : link – Smaragdine

Community Edition doesn't support databricks-connect functionality. It's only available for the enterprise version. This is the reason there is a token generation available for community edition. – Goethite

@RohitMishra did you mean "there is a no token generation available for community edition," as stated in your answer? – Redeemable

@KBurchfiel, Yes. Last time when I checked that was the case. – Goethite

Answer 3

This article "Azure Databricks from PyCharm IDE", describes how to connect Databricks cluster from PyCharm IDE.

Hope this helps. Do let us know if you have any queries.

Loran

I am asking for connecting to Databricks Community Edition Cluster. – Bang

Where do I get a User Token for Community edition of Databricks? – Hirai