what is the cluster manager used in Databricks ? How do I change the number of executors in Databricks clusters? [closed]

Question

What is the cluster manager used in Databricks? How do I change the number of executors in Databricks clusters ?

Armistead

Answer 1

To answer the question:

What is the cluster manager used in Databricks?

I try to dig this information out, but I couldn't find any info about it from the official docs.

It seems like Databricks is not using any of the cluster managers from Spark mentioned here

According to this presentation, On page 23, it mentions 3 parts of Databricks cluster manager

• Instance manager
• Resource manager
• Spark Cluster manager

So I guess Databricks uses its own pripriotory cluster manager.

Lefkowitz

I really hope databricks one day will release this info. – Stuartstub