Spark SQL Inner join is the default join in and it’s mostly used, this joins two DataFrame/Datasets on key columns, where keys don’t match the rows get dropped from both datasets.…

In this article, I will explain Spark SQL Self Join (Joining DataFrame to itself) with Scala Example. Joins are not complete without a self join, though there is no self-join…

When you join two Spark DataFrames using Left Anti Join (left, left anti, left_anti), it returns only columns from the left DataFrame for non-matched records. In this Spark article, I will…

Spark Left Semi Join (semi, left semi, left_semi) is similar to inner join difference being left semi-join returns all columns from the left DataFrame/Dataset and ignores all columns from the right dataset. In other…

Spark SQL Right Outer Join returns all rows from the right DataFrame regardless of math found on the left DataFrame, when the join expression doesn’t match, it assigns null for…

In this Spark article, I will explain how to do Full Outer Join (outer, full,fullouter, full_outer) on two DataFrames with Scala Example and Spark SQL. Before we jump into Spark…