Spark – How to create an empty RDD?
We often need to create empty RDD in Spark, and empty RDD can be created in several ways, for example, with partition, without partition, and with pair RDD. In this…
3 Comments
November 21, 2019
Convert Spark RDD to DataFrame | Dataset
While working in Apache Spark with Scala, we often need to Convert Spark RDD to DataFrame and Dataset as these provide more advantages over RDD. For instance, DataFrame is a…
August 22, 2019
Photo by Deva Darshan on Unsplash
Different ways to create Spark RDD
Spark RDD can be created in several ways using Scala & Pyspark languages, for example, It can be created by using sparkContext.parallelize(), from text file, from another RDD, DataFrame, and…
February 4, 2019
Create a Spark RDD using Parallelize
Let's see how to create Spark RDD using parallelize with sparkContext.parallelize() method and using Spark shell and Scala example. Before we start let me explain what is RDD, Resilient Distributed…
December 4, 2018