Spark Enable Hive Support
Let's discuss how to enable hive support in Spark pr PySpark to work with Hive in order to read and write. In this article, I will explain Spark configurations to…
0 Comments
February 21, 2023
Why Hive Table is loading with NULL values?
Problem: I created a new managed table in Hive and tried loading the data to the table, post LOAD command, select shows all NULL values as the data got loaded…
January 3, 2023
PySpark SQL Read Hive Table
How to read a Hive table into PySpark DataFrame? PySpark SQL supports reading a Hive table to DataFrame in two ways: the SparkSesseion.read.table() method and the SparkSession.sql() statement. In order…
December 10, 2022
Spark SQL Read Hive Table
How to read a Hive table into Spark DataFrame? Spark SQL supports reading a Hive table to DataFrame in two ways: the spark.read.table() method and the spark.sql() statement. spark.read is…
December 10, 2022
How to Connect Spark to Remote Hive
When you are working with Spark and Hive you would be required to connect Spark to the remote hive cluster. In my last article, I explained how to write Spark…
December 9, 2022
PySpark AnalysisException: Hive support is required to CREATE Hive TABLE (AS SELECT);
While I am trying to create a Hive Table from PySpark I was getting an error AnalysisException: Hive support is required to CREATE Hive TABLE (AS SELECT); from os.path import…
December 3, 2022
PySpark Save DataFrame to Hive Table
To save a PySpark DataFrame to Hive table use saveAsTable() function or use SQL CREATE statement on top of the temporary view. In order to save DataFrame as a Hive…
December 3, 2022
Solved: Unable to instantiate SparkSession with Hive support because Hive classes are not found
While running a Spark application with Hive enabled getting the below error message? Exception in thread "main" java.lang.IllegalArgumentException: Unable to instantiate SparkSession with Hive support because Hive classes are not…
December 2, 2022
Spark Save DataFrame to Hive Table
How to save or write a Spark DataFrame to a Hive table? Spark SQL supports writing DataFrame to Hive tables, there are two ways to write a DataFrame as a…
December 2, 2022
Spark SQL Create a Table
Spark & PySpark SQL allows you to create a database and table either directly from DataFrame, from Temporary views, or from external source files. In this article, we shall discuss…
December 2, 2022
- 1
- 2
- 3
- 4
- 5
- Go to the next page
