How to use Scala implicit class in Java

Question

I have a Scala Implicit class from RecordService API, which i wanted to use in Java file.

package object spark {

   implicit class RecordServiceContext(ctx: SparkContext) {
     def recordServiceTextFile(path: String) : RDD[String] = {
      new RecordServiceRDD(ctx).setPath(path)
          .map(v => v(0).asInstanceOf[Text].toString)
    }
  }

}

Now i am trying to import this in a Java file using below import.

import com.cloudera.recordservice.spark.*;

But i am not able to use recordServiceTextFile("path") from sparkContext.

In Scala the import is little different and its working.

Smiley

I don't see an implicit class. – Hamford

Importing Scala code in Java code is not a trivial operation to my eyes. It requires some understanding of the javac internals. Also the question is not clear. Can you be a but more clear on the question ? The more "full" the question the better the answer. – Widmer

@Ramesh You're welcome but that wasn't me. – Hamford

Answer 1

Here is simple definition of implicit class in package object

package object spark {
  implicit class Ext(param: Int) {
    def a = param + 1
  }
}

and here is how you can use it from java

public class Test {
    public static void main(String[] args) {
        spark.package$.MODULE$.Ext(123).a();
    }
}

so you can basically use RecordServiceContext as a method that wraps your SparkContext and adds an extra method that you can call. That is optimization for implicit classes.

That would be something like this:

SparkContext c = ???
RDD<String> rdd = com.cloudera.recordservice.spark.package$.MODULE$.RecordServiceContext(c)
   .recordServiceTextFile("asdf");

Nympho

Answer 2

A package object spark is compiled to a class package in the package spark. The implicit class RecordServiceContext will get compiled to a static method RecordServiceContext (that's scala's implicit def) in package and a class package$RecordServiceContext.

So the following code should do it:

import com.cloudera.recordservice.spark.*;

//some code

RDD<String> rdd = package.RecordServiceContext(myContext).recordServiceTextFile(pathToFile);

//some code

But package is probably a reserved keyword, and Java has no way of escaping them as far as I know. So you'll have to do some reflection to invoke the RecordServiceContext method.

Hamford

Answer 3

 SparkContext ctx = ...
 RecordServiceContext rsct = new RecordServiceContext(ctx)
 recordServiceTextFile("/your_path")

Tribesman

Answer 4

This should do it.

String s = new spark.RecordServiceContext("safa").recordServiceTextFile("dsf");

I changed the signatures though.

My Scala class looks like this ,

object spark {
implicit class RecordServiceContext(ctx: String) {
 def recordServiceTextFile(path: String) : String = {
 "test"
}
}
}

My java class looks like this,

 public class TestScalaCall {
public static void main(String args[]){
    String s = new spark.RecordServiceContext("safa").recordServiceTextFile("dsf");
}
}

Edit ---

So a quick look of the Scala change requests shows us this.

They are actually working on making a class defined under a package object behave the same way as defining it inside a regular package. But that is targeted for the yet to be release 2.12 .

So the recommendation they are giving is keep only absolutely necessary classes/objects that do not needs any external interaction inside package objects. Otherwise keep them under regular packages. So for now you need to not use the package object construct.

Also , a point worth pondering "Does it really make sense to define something that is accessible on the outside inside a package object ? "

Widmer

your implicit class missing package before object spark, that's the main problem i am facing... – Smiley

i added the explanation in the answer. That is something that the scala team is working on. – Widmer