PySpark – Loop/Iterate Through Rows in DataFrame
PySpark provides map(), mapPartitions() to loop/iterate through rows in RDD/DataFrame to perform the complex transformations,…
2 Comments
March 27, 2021
Spark map() vs flatMap() with Examples
What is the difference between Spark map() vs flatMap() is a most asked interview question,…
March 11, 2021
Spark map() Transformation
Spark map() is a transformation operation that is used to apply the transformation on every…
August 22, 2020
PySpark map() Transformation
In PySpark, the map() transformation is used to apply a function/lambda to each element of…
Comments Off on PySpark map() Transformation
August 22, 2020
PySpark RDD Transformations with examples
PySpark RDD Transformations are lazy evaluation and is used to transform/update from one RDD into…
August 22, 2020
Spark map() vs mapPartitions() with Examples
Spark map() and mapPartitions() transformations apply the function on each element/record/row of the DataFrame/Dataset and…
July 18, 2020
Photo by Chris Lawton on Unsplash
Spark RDD Transformations with examples
RDD Transformations are Spark operations when executed on RDD, it results in a single or…
November 30, 2019
Spark SQL Map functions – complete list
In this article, I will explain the usage of the Spark SQL map functions map(), map_keys(), map_values(), map_contact(), map_from_entries() on DataFrame…
October 6, 2019