Skip to content
  • Home
  • About
|       { One stop for all Spark Examples }
Spark by {Examples}
  • Spark
    • Spark RDD
    • Spark DataFrame
    • Spark SQL Functions
    • What’s New in Spark 3.0?
    • Spark Streaming
  • PySpark
  • Pandas
  • Hive
  • Kafka
  • H2O.ai
  • More
    • Apache Hadoop
    • Apache HBase
    • Apache Cassandra
    • Snowflake Database
    • H2O Sparkling Water
    • Scala Language
    • Python NumPy
    • FAQ’s
Menu Close
  • Spark
    • Spark RDD
    • Spark DataFrame
    • Spark SQL Functions
    • What’s New in Spark 3.0?
    • Spark Streaming
  • PySpark
  • Pandas
  • Hive
  • Kafka
  • H2O.ai
  • More
    • Apache Hadoop
    • Apache HBase
    • Apache Cassandra
    • Snowflake Database
    • H2O Sparkling Water
    • Scala Language
    • Python NumPy
    • FAQ’s
  • Home
  • About
Read more about the article PySpark – Distinct to Drop Duplicate Rows
PySpark

PySpark – Distinct to Drop Duplicate Rows

PySpark distinct() function is used to drop/remove the duplicate rows (all columns) from DataFrame and dropDuplicates() is used to drop rows based on selected (one or multiple) columns. In this…

5 Comments
August 12, 2020
Read more about the article Spark SQL – How to Remove Duplicate Rows
Apache Spark

Spark SQL – How to Remove Duplicate Rows

Duplicate rows could be remove or drop from Spark SQL DataFrame using distinct() and dropDuplicates() functions, distinct() can be used to remove rows that have the same values on all…

4 Comments
December 25, 2019
Read more about the article Spark SQL – Get Distinct Multiple Columns
Apache Spark

Spark SQL – Get Distinct Multiple Columns

In this Spark SQL tutorial, you will learn different ways to get the distinct values in every column or selected multiple columns in a DataFrame using methods available on DataFrame…

2 Comments
December 24, 2019

Categories

  • Apache Hadoop
  • Apache Spark
  • Apache Spark Streaming
  • Apache Kafka
  • Apache HBase
  • Apache Cassandra
  • Snowflake Database
  • H2O Sparkling Water
  • PySpark

Recent Posts

  • Python Dictionary Methods
  • Python Dictionary setdefault() Method
  • Python Dictionary popitem() Method
  • Python Dictionary pop() Method
  • Python Dictionary update() Method
  • Python Dictionary fromkeys() Usage With Example
  • Python Dictionary copy()
  • Python Dictionary keys() Method Usage
  • Python Dictionary Values()
Privacy & Cookies: This site uses cookies. By continuing to use this website, you agree to their use.
To find out more, including how to control cookies, see here: Cookie Policy

About SparkByExamples.com

SparkByExamples.com is a Big Data and Spark examples community page, all examples are simple and easy to understand, and well tested in our development environment Read more ..
Copyright sparkbyexamples.com