Spark Schema defines the structure of the DataFrame which you can get by calling printSchema() method on the DataFrame object. Spark SQL provides StructType & StructField classes to programmatically specify the schema. By default, Spark infers the schema from the data, however, sometimes we may need to define our own…

Spark SQL StructType & StructField classes are used to programmatically specify the schema to the DataFrame and creating complex columns like nested struct, array and map columns. StructType is a collection of StructField's. Using StructField we can define column name, column data type, nullable column (boolean to specify if the…