Can we do Spark DataFrame transpose or pivot without aggregation?

off course you can, but unfortunately, you can't achieve using Pivot function. However, pivoting or transposing DataFrame structure without aggregation from rows to columns and columns to rows can be easily done using Spark and Scala hack. please refer to stackoverflow example.

How to Pivot and Unpivot a Spark Data Frame

| *** Please Subscribe for Ad Free & Premium Content ***

Post author:Naveen Nelamali
Post category:Apache Spark / Member
Post last modified:April 24, 2024
Reading time:8 mins read

You are currently viewing How to Pivot and Unpivot a Spark Data Frame

Spark Pivot Table DataFrame

Access to this content is reserved for our valued members. Please log in to your account to unlock this exclusive material.

Tags: groupby, pivot, stack, transponse, unpivot