How to convert RDD of Avro's GenericData.Record to DataFrame?

Question

Perhaps this question may seem a bit abstract, here it is:

val originalAvroSchema : Schema   = // read from a file
val rdd : RDD[GenericData.Record] = // From some streaming source

// Looking for a handy:
val df: DataFrame   = rdd.toDF(schema)

I explore spark-avro but it has support only to read from a file, not from existing RDD.

Deci

not sure why the answer was deleted – Softcover

there is a pull request for what you are looking for github.com/databricks/spark-avro/pull/113/files – Softcover

I deleted the answer because it had been downvoted. – Otte

Answer 1

import com.databricks.spark.avro._

val sqlContext = new SQLContext(sc)
val rdd : RDD[MyAvroRecord] = ...
val df = rdd.toAvroDF(sqlContext)

Griselgriselda