How to connect Zeppelin to Spark 1.5 built from the sources?

Question

I pulled the latest source from the Spark repository and built locally. It works great from an interactive shell like spark-shell or spark-sql.

Now I want to connect Zeppelin to my Spark 1.5, according to this install manual. I published the custom Spark build to the local maven repository and set the custom Spark version in the Zeppelin build command. The build process finished successfully but when I try to run basic things like sc inside notebook, it throws:

akka.ConfigurationException: Akka JAR version [2.3.11] does not match the provided config version [2.3.4]

Version 2.3.4 is set in pom.xml and spark/pom.xml, but simply changing them won’t even let me get a build.

If I rebuild Zeppelin with the standard -Dspark.vesion=1.4.1, everything works.

Landowska

Hi, I see you're new to StackOverflow. If you feel an answer solved the problem, please mark it as 'accepted' by clicking the green check mark. This helps keep the focus on older SO which still don't have answers. – Apocynaceous

Answer 1

Update 2016-01

Spark 1.6 support has landed to master and is available under -Pspark-1.6 profile.

---

Update 2015-09

Spark 1.5 support has landed to master and is available under -Pspark-1.5 profile.

---

Work on supporting Spark 1.5 in Apache Zeppelin (incubating) was done under this PR apache/incubator-zeppelin#269 which will lend to master soon.

For now, building from Spark_1.5 branch with -Pspark-1.5 should do the trick.

Apocynaceous