spark build path is cross-compiled with an incompatible version of Scala (2.10.0)

Question

When i try to execute sparksql code in scala IDE im getting below error,Could anyone help me to sort out this please?

spark build path is cross-compiled with an incompatible version of Scala (2.10.0). In case this report is mistaken, this check can be disabled in the compiler preference page. spark       Unknown Scala Version Problem
akka-remote_2.10-2.3.11.jar of spark build path is cross-compiled with an incompatible version of Scala (2.10.0). In case this report is mistaken, this check can be disabled in the compiler preference page.  spark       Unknown Scala Version Problem
akka-slf4j_2.10-2.3.11.jar of spark build path is cross-compiled with an incompatible version of Scala (2.10.0). In case this report is mistaken, this check can be disabled in the compiler preference page.   spark       Unknown Scala Version Problem
breeze_2.10-0.11.2.jar of spark build path is cross-compiled with an incompatible version of Scala (2.10.0). In case this report is mistaken, this check can be disabled in the compiler preference page.   spark       Unknown Scala Version Problem
breeze-macros_2.10-0.11.2.jar of spark build path is cross-compiled with an incompatible version of Scala (2.10.0). In case this report is mistaken, this check can be disabled in the compiler preference page.    spark       Unknown Scala Version Problem
chill_2.10-0.5.0.jar of spark build path is cross-compiled with an incompatible version of Scala (2.10.0). In case this report is mistaken, this check can be disabled in the compiler preference page. spark       Unknown Scala Version Problem
jackson-module-scala_2.10-2.4.4.jar of spark build path is cross-compiled with an incompatible version of Scala (2.10.0). In case this report is mistaken, this check can be disabled in the compiler preference page.  spark       Unknown Scala Version Problem
json4s-ast_2.10-3.2.10.jar of spark build path is cross-compiled with an incompatible version of Scala (2.10.0). In case this report is mistaken, this check can be disabled in the compiler preference page.   spark       Unknown Scala Version Problem
json4s-core_2.10-3.2.10.jar of spark build path is cross-compiled with an incompatible version of Scala (2.10.0). In case this report is mistaken, this check can be disabled in the compiler preference page.  spark       Unknown Scala Version Problem
json4s-jackson_2.10-3.2.10.jar of spark build path is cross-compiled with an incompatible version of Scala (2.10.0). In case this report is mistaken, this check can be disabled in the compiler preference page.   spark       Unknown Scala Version Problem
quasiquotes_2.10-2.0.1.jar of spark build path is cross-compiled with an incompatible version of Scala (2.10.0). In case this report is mistaken, this check can be disabled in the compiler preference page.   spark       Unknown Scala Version Problem
scalatest_2.10-2.2.5.jar of spark build path is cross-compiled with an incompatible version of Scala (2.10.0). In case this report is mistaken, this check can be disabled in the compiler preference page. spark       Unknown Scala Version Problem
spark-catalyst_2.10-1.4.0.jar of spark build path is cross-compiled with an incompatible version of Scala (2.10.0). In case this report is mistaken, this check can be disabled in the compiler preference page.    spark       Unknown Scala Version Problem
spark-core_2.10-1.5.2-tests.jar of spark build path is cross-compiled with an incompatible version of Scala (2.10.0). In case this report is mistaken, this check can be disabled in the compiler preference page.  spark       Unknown Scala Version Problem
spark-core_2.10-1.5.2.jar of spark build path is cross-compiled with an incompatible version of Scala (2.10.0). In case this report is mistaken, this check can be disabled in the compiler preference page.    spark       Unknown Scala Version Problem
spark-graphx_2.10-1.4.0.jar of spark build path is cross-compiled with an incompatible version of Scala (2.10.0). In case this report is mistaken, this check can be disabled in the compiler preference page.  spark       Unknown Scala Version Problem
spark-launcher_2.10-1.5.2.jar of spark build path is cross-compiled with an incompatible version of Scala (2.10.0). In case this report is mistaken, this check can be disabled in the compiler preference page.    spark       Unknown Scala Version Problem
spark-mllib_2.10-1.4.0-tests.jar of spark build path is cross-compiled with an incompatible version of Scala (2.10.0). In case this report is mistaken, this check can be disabled in the compiler preference page. spark       Unknown Scala Version Problem
spark-mllib_2.10-1.4.0.jar of spark build path is cross-compiled with an incompatible version of Scala (2.10.0). In case this report is mistaken, this check can be disabled in the compiler preference page.   spark       Unknown Scala Version Problem
spark-network-common_2.10-1.5.2.jar of spark build path is cross-compiled with an incompatible version of Scala (2.10.0). In case this report is mistaken, this check can be disabled in the compiler preference page.  spark       Unknown Scala Version Problem
spark-network-shuffle_2.10-1.5.2.jar of spark build path is cross-compiled with an incompatible version of Scala (2.10.0). In case this report is mistaken, this check can be disabled in the compiler preference page. spark       Unknown Scala Version Problem
spark-sql_2.10-1.4.0.jar of spark build path is cross-compiled with an incompatible version of Scala (2.10.0). In case this report is mistaken, this check can be disabled in the compiler preference page. spark       Unknown Scala Version Problem
spark-streaming_2.10-1.5.2.jar of spark build path is cross-compiled with an incompatible version of Scala (2.10.0). In case this report is mistaken, this check can be disabled in the compiler preference page.   spark       Unknown Scala Version Problem
spark-unsafe_2.10-1.5.2.jar of spark build path is cross-compiled with an incompatible version of Scala (2.10.0). In case this report is mistaken, this check can be disabled in the compiler preference page.  spark       Unknown Scala Version Problem
spire_2.10-0.7.4.jar of spark build path is cross-compiled with an incompatible version of Scala (2.10.0). In case this report is mistaken, this check can be disabled in the compiler preference page. spark       Unknown Scala Version Problem
spire-macros_2.10-0.7.4.jar of spark build path is cross-compiled with an incompatible version of Scala (2.10.0). In case this report is mistaken, this check can be disabled in the compiler preference page.  spark       Unknown Scala Version Problem
The version of scala library found in the build path of spark (2.10.4) is prior to the one provided by scala IDE (2.11.7). Setting a Scala Installation Choice to match.    spark       Unknown Scala Version Problem

Cloudcapped

See this similar yet different question: #47207497 – Consolation

Answer 1

In your project you are using jars built with different Scala versions; indeed from the log:

• Scala IDE uses Scala 2.11.7
• Apache Spark 1.5.2 built with Scala 2.10

You need to align the jars versions. Actually Spark 1.5.2 pre-build (downloadable from here) has built with Scala 2.10 for compatibility reasons (see here). On the web site there is a note:

Note: Scala 2.11 users should download the Spark source package and build with Scala 2.11 support.

In order to solve the issue, in your project use the same Scala version used to build Spark.

I suggest to try to switch to Scala 2.10 in your Scala IDE and it will solve the issue.

Eclipse + Maven

In my case, I'm using Eclipse with Scala IDE and Maven so I updated the Maven dependencies in this way:

<dependency>
    <groupId>org.scala-lang</groupId>
    <artifactId>scala-library</artifactId>
    <version>2.10.6</version>
</dependency>
<dependency>
    <groupId>org.apache.spark</groupId>
    <artifactId>spark-core_2.10</artifactId>
    <version>1.5.2</version>
</dependency>

Then, I changed the Scala version in the IDE: Right click on the project -> Scala -> set Scala installation or Right Click on the Scala Library Container -> Properties and choose the Scala 2.10 as shown below

Endorsement

Scala-2.10 has no more planned maintenance. There is good reason to get a project working with Spark and Scala-2.11 instead. – Cardoza

I need to find a way to to fix this automatically. I have over 20 projects in my workspace and don't want to manually set the Scala Compiler version for every project. – Consolation

Please see this similar yet different question: #47207497 – Consolation

Answer 2

Click right on your project, then select Properties. Select Scala Compiler Click "Use Project Setting" Select Scala Installation select your scala version, for example, 2.11.11.

Gymnast

Answer 3

I had same problem. How I solved it. Scala embedded Eclipse -> Project -> Right Click -> Project -> Scala Compiler -> Right side you will be getting list of Scala Installation-> Here choose Latest 2.10 bundle (dynamic).

I have tested and it worked fine for me.

• Binod Suman

Baal

Answer 4

"-Xsource:2.11 -Ymacro-expand:none" solved my problem , just updated in right click project -> properties -> Scala Compiler , Additional Command Line Parameter , to fix this "build path is cross-compiled with an incompatible version of Scala (2.11.0). In case this report is mistaken, this check can be disabled in the compiler preference page."

Tapp