Spark ml and PMML export

Question

I know that it's possible to export models as PMML with Spark-MLlib, but what about Spark-ML?

Is it possible to convert LinearRegressionModel from org.apache.spark.ml.regression to a LinearRegressionModel from org.apache.spark.mllib.regression to be able to invoke the toPMML() method?

Circumpolar

Answer 1

You can convert Spark ML pipelines to PMML using the JPMML-SparkML library:

StructType schema = dataFrame.schema()
PipelineModel pipelineModel = pipeline.fit(dataFrame);
org.dmg.pmml.PMML pmml = org.jpmml.sparkml.ConverterUtil.toPMML(schema, pipelineModel);
JAXBUtil.marshalPMML(pmml, new StreamResult(System.out));

Ailbert

Thanks I'll try it asap – Circumpolar