Disable hive support in sparklyr

Question

Is there any way to disable the hive support in sparklyr?

Just like in SparkR:

sparkR.session(master="local[*]", enableHiveSupport=FALSE)

Springy

Answer 1

That option was introduced in sparklyr 1.3.0 after breaking Ron's answer solution on version 1.2.0 as described on issue #2460.

library(sparklyr)

config <- spark_config()
config$sparklyr.connect.enablehivesupport <- FALSE

sc <- spark_connect(master = "local", config = config)

Seethe

Answer 2

You can disable Hive in sparklyr by setting the SQL catalog implementation to in-memory.

# get the default config
conf <- spark_config()
# set the catalog implementation, defaults to hive, but we want it in-memory
conf$spark.sql.catalogImplementation <- "in-memory"
sc <- spark_connect(master = "local", config = conf)

Allotrope