Spark Epoch time to timestamp and Date

In this Spark article, you will learn how to convert or cast Epoch time to Timestamp and Date using SQL function from_unixtime() and Scala language

What is Epoch Time

Epoch time is also known as Unix time which specifies the moment in time since 1970-01-01 00:00:00 UTC. It is the number of seconds passed since Epoch time. Epoch time is widely used in Unix like operating systems.

Convert Epoch time to timestamp

from_unixtime() SQL function is used to convert or cast Epoch time to timestamp string and this function takes Epoch time as a first argument and formatted string time as the second argument. As a first argument, we use unix_timestamp() which returns the current timestamp in Epoch time (Long) as an argument.


  //Convert unix timestamp to timestamp
  val timeDF = Seq(1).toDF("seq").select(
    from_unixtime(unix_timestamp()).as("timestamp_1"),
    from_unixtime(unix_timestamp(),"MM-dd-yyyy HH:mm:ss").as("timestamp_2"),
    from_unixtime(unix_timestamp(),"dd-MM-yyyy HH:mm:ss").as("timestamp_3"),
    from_unixtime(unix_timestamp(),"HH:mm:ss").as("timestamp_4")
  ).show()

Yields below output


+-------------------+-------------------+-------------------+-----------+
|        timestamp_1|        timestamp_2|        timestamp_3|timestamp_4|
+-------------------+-------------------+-------------------+-----------+
|2019-12-24 10:36:12|12-24-2019 10:36:12|24-12-2019 10:36:12|   10:36:12|
+-------------------+-------------------+-------------------+-----------+

Convert Epoch time to Date

Similar to above, here we use the same from_unixtime() SQL function to convert Epoch time to date besides we need to provide formatted string date as a second argument to the function


  //Convert unix timestamp to date
  val dateDF = Seq(1).toDF("seq").select(
    from_unixtime(unix_timestamp(),"MM-dd-yyyy").as("date_1"),
    from_unixtime(unix_timestamp(),"dd-MM-yyyy HH:mm:ss").as("date_2"),
    from_unixtime(unix_timestamp(),"yyyy-MM-dd").as("date_3")
  ).show(false)

Yields below output


+----------+-------------------+----------+
|date_1    |date_2             |date_3    |
+----------+-------------------+----------+
|12-24-2019|24-12-2019 10:36:12|2019-12-24|
+----------+-------------------+----------+

The complete example is available at GitHub project for reference

Conclusion

In this Spark article, you have learned how to convert or cast the Epoch time to Date or Timestamp using from_unixtime() function along with Scala example.

NNK

SparkByExamples.com is a Big Data and Spark examples community page, all examples are simple and easy to understand and well tested in our development environment Read more ..

Leave a Reply

You are currently viewing Spark Epoch time to timestamp and Date