Problem: How to convert the Spark Timestamp column to String on DataFrame column?
Solution: Using <em>date_format</em>() Spark SQL date function, we can convert Timestamp to the String format. Spark support all Java Data formatted patterns for conversion. In this article, we will see a few examples in the Scala language.
1. Complete example of converting Timestamp to String
In this example, I am using Spark current_timestamp() to get the current system timestamp and then we convert this to different string patterns.
package com.sparkbyexamples.spark.dataframe.functions.datetime
import org.apache.spark.sql.SparkSession
import org.apache.spark.sql.functions.{current_date, current_timestamp, date_format}
object TimestampToString extends App {
val spark:SparkSession = SparkSession.builder()
.master("local")
.appName("SparkByExamples.com")
.getOrCreate()
spark.sparkContext.setLogLevel("ERROR")
import spark.sqlContext.implicits._
Seq(1).toDF("seq").select(
current_timestamp().as("current_date"),
date_format(current_timestamp(),"yyyy MM dd").as("yyyy MM dd"),
date_format(current_timestamp(),"MM/dd/yyyy hh:mm").as("MM/dd/yyyy"),
date_format(current_timestamp(),"yyyy MMM dd").as("yyyy MMMM dd"),
date_format(current_timestamp(),"yyyy MMMM dd E").as("yyyy MMMM dd E")
).show(false)
}
Yields below output:
// Output:
+------------+----------+----------------+------------+--------------------+
|current_date|yyyy MM dd|MM/dd/yyyy |yyyy MMMM dd|yyyy MMMM dd E |
+------------+----------+----------------+------------+--------------------+
|2019-11-17 |2019 11 17|11/17/2019 02:44|2019 Nov 17 |2019 November 17 Sun|
+------------+----------+----------------+------------+--------------------+
Happy Learning !!