PySpark – Loop/Iterate Through Rows in DataFrame
PySpark provides map(), mapPartitions() to loop/iterate through rows in RDD/DataFrame to perform the complex transformations,…
2 Comments
March 27, 2021
Spark – Extract DataFrame Column as List
Let's see how to convert/extract the Spark DataFrame column as a List (Scala/Java Collection), there…
March 15, 2021
Collect() – Retrieve data from Spark RDD/DataFrame
Spark collect() and collectAsList() are action operation that is used to retrieve all the elements…
August 11, 2020
PySpark Collect() – Retrieve data from DataFrame
PySpark RDD/DataFrame collect() is an action operation that is used to retrieve all the elements…
August 11, 2020