You are currently viewing Spark date_format() – Convert Timestamp to String

Problem: How to convert the Spark Timestamp column to String on DataFrame column?

Solution: Using <em>date_format</em>() Spark SQL date function, we can convert Timestamp to the String format. Spark support all Java Data formatted patterns for conversion. In this article, we will see a few examples in the Scala language.

1. Complete example of converting Timestamp to String

In this example, I am using Spark current_timestamp() to get the current system timestamp and then we convert this to different string patterns.

package com.sparkbyexamples.spark.dataframe.functions.datetime

import org.apache.spark.sql.SparkSession
import org.apache.spark.sql.functions.{current_date, current_timestamp, date_format}

object TimestampToString extends App {

  val spark:SparkSession = SparkSession.builder()

  import spark.sqlContext.implicits._

    date_format(current_timestamp(),"yyyy MM dd").as("yyyy MM dd"),
    date_format(current_timestamp(),"MM/dd/yyyy hh:mm").as("MM/dd/yyyy"),
    date_format(current_timestamp(),"yyyy MMM dd").as("yyyy MMMM dd"),
    date_format(current_timestamp(),"yyyy MMMM dd E").as("yyyy MMMM dd E")


Yields below output:

// Output:
|current_date|yyyy MM dd|MM/dd/yyyy      |yyyy MMMM dd|yyyy MMMM dd E      |
|2019-11-17  |2019 11 17|11/17/2019 02:44|2019 Nov 17 |2019 November 17 Sun|

Happy Learning !!

Naveen Nelamali

Naveen Nelamali (NNK) is a Data Engineer with 20+ years of experience in transforming data into actionable insights. Over the years, He has honed his expertise in designing, implementing, and maintaining data pipelines with frameworks like Apache Spark, PySpark, Pandas, R, Hive and Machine Learning. Naveen journey in the field of data engineering has been a continuous learning, innovation, and a strong commitment to data integrity. In this blog, he shares his experiences with the data as he come across. Follow Naveen @ LinkedIn and Medium