How to do map and reduce in SparkR

Question

How do I do map and reduce operations using SparkR? All I can find is stuff about SQL queries. Is there a way to do map and reduce using SQL?

Infielder

Not sure what you mean by doing "map and reduce using SQL," but this this tutorial should help you get a sense of how to write map/reduce in SparkR: blog.rstudio.org/2015/05/28/sparkr-preview-by-vincent-warmerdam – Mitchiner

Answer 1

See Writing R data frames returned from SparkR:::map for an example (the question itself). In short, the blog post referred to by sph21 is out of date. As of the current date, both map and reduce have been hidden in SparkR as private methods - there are open tickets to resolve that issue.

Affectionate