Spark Timestamp – Extract hour, minute and second

Problem: How to extract or get an hour, minute and second from a Spark timestamp column?

Solution: Spark functions provides hour(), minute() and second() functions to extract hour, minute and second from Timestamp column respectively.

hour – function hour() extracts hour unit from Timestamp column or string column containing a timestamp.


Syntax : hour(e: Column): Column

minute – function minute() extracts minute unit from Timestamp column or string column containing a timestamp.


Syntax : minute(e: Column): Column

second – function second() extracts the second unit from the Timestamp column or string column containing a timestamp.


Syntax : second(e: Column): Column

Extract hour, minute and second example


package com.sparkbyexamples.spark.dataframe.functions.datetime

import org.apache.spark.sql.SparkSession
import org.apache.spark.sql.functions.{col,hour,minute,second}

object GetTimeFromTimestamp extends App {

  val spark:SparkSession = SparkSession.builder()
    .master("local")
    .appName("SparkByExamples.com")
    .getOrCreate()
  spark.sparkContext.setLogLevel("ERROR")

  import spark.sqlContext.implicits._

  val df = Seq(("2019-07-01 12:01:19.000"),
    ("2019-06-24 12:01:19.000"),
    ("2019-11-16 16:44:55.406"),
    ("2019-11-16 16:50:59.406")).toDF("input_timestamp")

  df.withColumn("hour", hour(col("input_timestamp")))
    .withColumn("minute", minute(col("input_timestamp")))
    .withColumn("second", second(col("input_timestamp")))
    .show(false)

}

Yields below output


+-----------------------+----+------+------+
|input_timestamp        |hour|minute|second|
+-----------------------+----+------+------+
|2019-07-01 12:01:19.000|12  |1     |19    |
|2019-06-24 12:01:19.000|12  |1     |19    |
|2019-11-16 16:44:55.406|16  |44    |55    |
|2019-11-16 16:50:59.406|16  |50    |59    |
+-----------------------+----+------+------+

The complete code can be downloaded from GitHub

Related: How to add hours, minutes and timestamps

Happy Learning !!

Naveen (NNK)

Naveen (NNK) is a Data Engineer with 20+ years of experience in transforming data into actionable insights. Over the years, He has honed his expertise in designing, implementing, and maintaining data pipelines with frameworks like Apache Spark, PySpark, Pandas, R, Hive and Machine Learning. Naveen journey in the field of data engineering has been a continuous learning, innovation, and a strong commitment to data integrity. In this blog, he shares his experiences with the data as he come across. Follow Naveen @ @ LinkedIn

Leave a Reply

This Post Has 2 Comments

  1. kübrataş

    Very useful and clearly explained, thanks !

  2. kübrataş

    Very useful and clearly explained, thanks !