How to resolve No module named 'pyspark' Error in Jupyter notebook and any python editor? In python when you try to import PySpark library without installing or properly setting environment…
PySpark DataFrame DataFrame definition is very well explained by Databricks hence I do not want to define it again and confuse you. Below is the definition I took it from…
In this article, I will explain step-by-step how to do Apache Spark Installation on windows os 7, 10, and the latest version and also explain how to start a history…
Hive CAST(from_datatype as to_datatype) function is used to convert from one data type to another for example to cast String to Integer(int), String to Bigint, String to Decimal, Decimal to…
In this article, I will explain what is Hive Partitioning and Bucketing, the difference between Hive Partitioning vs Bucketing by exploring the advantages and disadvantages of each features with examples.…
In Hive, SHOW PARTITIONS command is used to show or list all partitions of a table from Hive Metastore, In this article, I will explain how to list all partitions,…
Hive ALTER TABLE command is used to update or drop a partition from a Hive Metastore and HDFS location (managed table). You can also manually update or drop a Hive…
pyspark.sql.functions provides a function split() to split DataFrame string Column into multiple columns. In this tutorial, you will learn how to split Dataframe single column into multiple columns using withColumn()…
Since Hive Version 0.14, Hive supports ACID transactions like delete and update records/rows on Table with similar syntax as traditional SQL queries. You need to enable Hive ACID support and…
To export a Hive table into a CSV file you can use either INSERT OVERWRITE DIRECTORY or by piping the output result of the select query into a CSV file.…