Spark date_format() – Convert Timestamp to String

Problem: How to convert the Spark Timestamp column to String on DataFrame column?

Solution: Using <em>date_format</em>() Spark SQL date function, we can convert Timestamp to the String format. Spark support all Java Data formatted patterns for conversion. In this article, we will see a few examples in the Scala language.

Complete example of converting Timestamp to String

In this example, I am using Spark current_timestamp() to get the current system timestamp and then we convert this to different string patterns.


package com.sparkbyexamples.spark.dataframe.functions.datetime

import org.apache.spark.sql.SparkSession
import org.apache.spark.sql.functions.{current_date, current_timestamp, date_format}

object TimestampToString extends App {

  val spark:SparkSession = SparkSession.builder()
    .master("local")
    .appName("SparkByExamples.com")
    .getOrCreate()
  spark.sparkContext.setLogLevel("ERROR")

  import spark.sqlContext.implicits._

  Seq(1).toDF("seq").select(
    current_timestamp().as("current_date"),
    date_format(current_timestamp(),"yyyy MM dd").as("yyyy MM dd"),
    date_format(current_timestamp(),"MM/dd/yyyy hh:mm").as("MM/dd/yyyy"),
    date_format(current_timestamp(),"yyyy MMM dd").as("yyyy MMMM dd"),
    date_format(current_timestamp(),"yyyy MMMM dd E").as("yyyy MMMM dd E")
  ).show(false)

}

Yields below output:


+------------+----------+----------------+------------+--------------------+
|current_date|yyyy MM dd|MM/dd/yyyy      |yyyy MMMM dd|yyyy MMMM dd E      |
+------------+----------+----------------+------------+--------------------+
|2019-11-17  |2019 11 17|11/17/2019 02:44|2019 Nov 17 |2019 November 17 Sun|
+------------+----------+----------------+------------+--------------------+

Happy Learning !!

NNK

SparkByExamples.com is a Big Data and Spark examples community page, all examples are simple and easy to understand and well tested in our development environment Read more ..

Leave a Reply