I am trying to run on of my scala file from spark-shell. This file calling some other jar files which have already been loaded into spark-context The problem is if something fails, it only prints...

The only two way I know to run Scala based spark code is to either compile a Scala program into a jar file and run it with spark-submit, or run a Scala script by using :load inside the spark-shell....

I have a Scala program that I want to execute using Spark shell, now when I copy paste into spark shell it doesn't work, I have to copy line by line inside. How should I copy all the program insid...

Need to execute the scala script through spark-shell with silent mode. When I am using spark-shell -i "file.scala", after the execution, I am getting into the scala interactive mode. I don't want t...

How to run a Spark-shell with hive.exec.dynamic.partition.mode=nonstrict? I try (as suggested here) export SPARK_MAJOR_VERSION=2; spark-shell --conf "hive.exec.dynamic.partition.mode=nonstrict" ...

Day_Date,timeofday_desc,Timeofday_hour,Timeofday_minute,Timeofday_second,value 2017-12-18,12:21:02 AM,0,21,2,“1.779209040E+08” 2017-12-19,12:21:02 AM,0,21,2,“1.779209040E+08” 2017-12-20,12:30...

When I was trying to implement an algorithm in Graphx with Scala, I didn't find it possible to activate all the vertices in the next ietration.. How can I send a message to all my graph vertices? ...

So i was trying to load the csv file inferring custom schema but everytime i end up with the following errors: Project_Bank.csv is not a Parquet file. expected magic number at tail [80, 65, 82, 49...