Object spark is not a member of package org

Question

When importing the following in Eclipse Scala-IDE

import org.apache.spark.SparkContext
import org.apache.spark.SparkContext._

I get this error: Object spark is not a member of package org

I installed the sbt-0.13.9.msi

What else should I do?

build.sbt

name := "scala-spark-app"

version := "1.0"

scalaVersion := "2.10.4"

libraryDependencies += "org.apache.spark" %% "spark-core" % "1.5.2"

Splenic

How did you create this project? Does the problem exist when you simply run sbt package? – Jamieson

It the error message literally "Object spark is not a member of package org"? Did you maybe somewhere type org.spark instead of org.apache.spark? – Savoury

C:\Users\Adham>sbt package Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=256m; support was removed in 8.0 [info] Set current project to adham (in build file:/C:/Users/Adham/) [success] Total time: 0 s, completed 27-Nov-2015 19:10:51 – Splenic

Why it is so complicated to use/setup Scala, setting up Java is very easy, just install eclipse and the JDK and you are done! – Splenic

Are you on mac or linux? Go to your project directory (where your build.sbt file lives), type sbt compile. Does everything work ok? – Encincture

Compling succeed but when I ran the .scala file I still get problems C:\Users\Adham\ScalaWorkSpace\Chapter1>sbt compile Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=256m; support was removed in 8.0 [info] Set current project to scala-spark-app (in build file:/C:/Users/Adham/ScalaWorkSpace/Chapter1/) [success] Total time: 5 s, completed 30-Nov-2015 12:25:30 – Splenic

in linux: adhem@adhem-ThinkPad-E520:~/workspace/Chapter1$ sbt compile [info] Loading project definition from /home/adhem/workspace/Chapter1/project/project [info] Loading project definition from /home/adhem/workspace/Chapter1/project [info] Set current project to scala-spark-app (in build file:/home/adhem/workspace/Chapter1/) [success] Total time: 0 s, completed Dec 1, 2015 11:13:39 PM – Splenic

in windows and in linux the problem is the same! – Splenic

Answer 1

I was receiving the same compilation error (on my Mac) with IntelliJ whereas sbt compile executed successfully. The following steps solved my problem:

1. sbt clean
2. sbt gen-idea (to rebuild IntelliJ project files)
3. quit and restart IntelliJ

I hope this helps.

Havstad

THis question is for Eclipse Scala-IDE and not IntelliJ – Goggles