Skip to content
  • Home
  • About
  • Write For US
|       { One stop for all Spark Examples }
Spark By {Examples}
  • Spark
    • Spark RDD
    • Spark DataFrame
    • Spark SQL Functions
    • What’s New in Spark 3.0?
    • Spark Streaming
    • Apache Spark Interview Questions
  • PySpark
  • Pandas
  • R
    • R Programming
    • R Data Frame
    • R dplyr Tutorial
    • R Data Frame
    • R Vector
    • R dplyr Tutorial
  • Snowflake
  • Hive
  • Interview Q
    • Spark Interview Questions
  • More
    • KafkaApache Kafka Tutorials with Examples
    • NumPy
    • H2O.ai
    • Apache Hadoop
    • Apache HBase
    • Apache Cassandra
    • H2O Sparkling Water
    • Scala Language
    • Python
Menu Close
  • Spark
    • Spark RDD
    • Spark DataFrame
    • Spark SQL Functions
    • What’s New in Spark 3.0?
    • Spark Streaming
    • Apache Spark Interview Questions
  • PySpark
  • Pandas
  • R
    • R Programming
    • R Data Frame
    • R dplyr Tutorial
    • R Data Frame
    • R Vector
    • R dplyr Tutorial
  • Snowflake
  • Hive
  • Interview Q
    • Spark Interview Questions
  • More
    • Kafka
    • NumPy
    • H2O.ai
    • Apache Hadoop
    • Apache HBase
    • Apache Cassandra
    • H2O Sparkling Water
    • Scala Language
    • Python
  • Home
  • About
  • Write For US
Read more about the article PySpark ArrayType Column With Examples
PySpark

PySpark ArrayType Column With Examples

PySpark pyspark.sql.types.ArrayType (ArrayType extends DataType class) is used to define an array data type column on DataFrame that holds the same type of elements, In this article, I will explain…

3 Comments
March 22, 2021
Read more about the article PySpark StructType & StructField Explained with Examples
PySpark

PySpark StructType & StructField Explained with Examples

PySpark StructType & StructField classes are used to programmatically specify the schema to the DataFrame and create complex columns like nested struct, array, and map columns. StructType is a collection…

3 Comments
August 13, 2020
Read more about the article Spark – Convert Array to Columns
Apache Spark

Spark – Convert Array to Columns

Problem: How to convert a DataFrame array to multiple columns in Spark? Solution: Spark doesn't have any predefined functions to convert the DataFrame array column to multiple columns however, we…

1 Comment
November 9, 2019
Read more about the article Spark Schema – Explained with Examples
Apache Spark

Spark Schema – Explained with Examples

Spark Schema defines the structure of the DataFrame which you can get by calling printSchema() method on the DataFrame object. Spark SQL provides StructType & StructField classes to programmatically specify…

4 Comments
November 8, 2019
Read more about the article Spark – Create a DataFrame with Array of Struct column
Apache Spark / Spark SQL Functions

Spark – Create a DataFrame with Array of Struct column

Problem: How to create a Spark DataFrame with Array of struct column using Spark and Scala? Using StructType and ArrayType classes we can create a DataFrame with Array of Struct…

0 Comments
October 30, 2019
Read more about the article Spark – Define DataFrame with Nested Array
Apache Spark / Spark SQL Functions

Spark – Define DataFrame with Nested Array

Problem: How to define Spark DataFrame using the nested array column (Array of Array)? Solution: Using StructType we can define an Array of Array (Nested Array) ArrayType(ArrayType(StringType)) DataFrame column using…

0 Comments
October 30, 2019
Read more about the article PySpark – explode nested array into rows
PySpark

PySpark – explode nested array into rows

Problem: How to explode & flatten nested array (Array of Array) DataFrame columns into rows using PySpark. Solution: PySpark explode function can be used to explode an Array of Array…

7 Comments
October 29, 2019
Read more about the article PySpark Explode Array and Map Columns to Rows
PySpark

PySpark Explode Array and Map Columns to Rows

In this article, I will explain how to explode array or list and map columns to rows using different PySpark DataFrame functions (explode(), explore_outer(), posexplode(), posexplode_outer()) with Python example. Before…

3 Comments
October 28, 2019
Read more about the article Spark Flatten Nested Array to Single Array Column
Apache Spark / Spark SQL Functions

Spark Flatten Nested Array to Single Array Column

Problem: How to flatten the Array of Array or Nested Array DataFrame column into a single array column using Spark. Solution: Spark SQL provides flatten function to convert an Array…

0 Comments
October 20, 2019
Read more about the article Spark Flatten Nested Array to Single Array Column
Apache Spark / Spark SQL Functions

Spark Flatten Nested Array to Single Array Column

Problem: How to flatten the Array of Array or Nested Array DataFrame column into a single array column using Spark. Solution: Spark SQL provides flatten function to convert an Array…

0 Comments
October 20, 2019
  • 1
  • 2
  • Go to the next page
Privacy & Cookies: This site uses cookies. By continuing to use this website, you agree to their use.
To find out more, including how to control cookies, see here: Cookie Policy

Top Tutorials

  • Apache Spark Tutorial
  • PySpark Tutorial
  • Python Pandas Tutorial
  • R Programming Tutorial
  • Python NumPy Tutorial
  • Apache Hive Tutorial
  • Apache HBase Tutorial
  • Apache Cassandra Tutorial
  • Apache Kafka Tutorial
  • Snowflake Data Warehouse Tutorial
  • H2O Sparkling Water Tutorial

Categories

  • Apache Spark
  • PySpark
  • Pandas
  • R Programming
  • Snowflake Database
  • NumPy
  • Apache Hive
  • Apache HBase
  • Apache Kafka
  • Apache Cassandra
  • H2O Sparkling Water

About SparkByExamples.com

SparkByExamples.com is a Big Data and Spark examples community page, all examples are simple and easy to understand, and well tested in our development environment Read more ..
  • Opens in a new tab
  • Opens in a new tab
  • Opens in a new tab
  • Opens in a new tab
  • Opens in a new tab
[email protected]
+1 (949) 345-0676
Desert Bloom
Irvine, CA 92618
USA
Copyright sparkbyexamples.com