Spark – Sort by column in descending order?

  • Post author:
  • Post category:Apache Spark
  • Post last modified:November 18, 2022

In order to sort by descending order in Spark DataFrame, we can use desc property of the Column class or desc() sql function. In this article, I will explain the sorting dataframe by using these approaches on multiple columns.

Using sort() for descending order

First, let’s do the sort.


Now, let’s do the sort using desc property of Column class and In order to get column class we use col() SQL function

import org.apache.spark.sql.functions.col

Finally, let’s see how desc() SQL function by importing org.apache.spark.sql.functions.desc

import org.apache.spark.sql.functions.desc

Using orderBy() for descending

Alternatively, we can also use orderBy() function of the DataFrame to sort by descending order. All examples explained with sort() also works here.

import org.apache.spark.sql.functions.col
import org.apache.spark.sql.functions.desc

Using SQL to sort

And, we can also use SQL expression to sort by descending order.

spark.sql(" select employee_name,desc('department'),desc('state'),salary,age,bonus from DEPT")

Happy Learning !!

NNK is a Big Data and Spark examples community page, all examples are simple and easy to understand and well tested in our development environment Read more ..

Leave a Reply