How to parse string and format dates on DataFrame
Spark SQL Dataframe functions example on getting current system date-time, formatting Date to a String pattern and parsing String to Date using Scala language.
Spark SQL Dataframe functions example on getting current system date-time, formatting Date to a String pattern and parsing String to Date using Scala language.
How to calculate the difference between two dates in days, months and years in Spark…
Spark SQL Dataframe example of converting different date formats from a single column to a standard date format using Scala language and Date and Time functions.
Spark SQL defines built-in standard String functions in DataFrame API, these String functions come in…
Spark SQL provides built-in standard Date and Timestamp (includes date and time) Functions defines in…
Spark SQL Function Introduction Spark SQL functions are a set of built-in functions provided by…
Let's see how to add a new column by assigning a literal or constant value to Spark DataFrame. Spark SQL provides lit() and typedLit() function to add a literal value to DataFrame. These both functions return Column type.
When you have a need to write complex XML structures from Spark Data Frame and Databricks XML API is not suitable for your use case, you could use XStream API to convert data to XML string and write it as a text. Let's see how to do this using an example.
Apache Spark can also be used to process or read simple to complex nested XML…
This article describes Spark Structured Streaming from Kafka in Avro file format and usage of…