Read more about the article Hive – HiveException java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient

Hive – HiveException java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient

I've installed Hive on Hadoop cluster and when running Hive commands from the hive shell, I am getting an error message HiveException java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient. Solution: You are…

0 Comments

October 3, 2020

Apache Spark / Apache Spark 3.0

Spark 3.0 Read Binary File into DataFrame

Since Spark 3.0, Spark supports a data source format binaryFile to read binary file (image, pdf, zip, gzip, tar e.t.c) into Spark DataFrame/Dataset. When used binaryFile format, the DataFrameReader converts the entire…

0 Comments

September 3, 2020

PySpark

PySpark Concatenate Columns

pyspark.sql.functions provides two functions concat() and concat_ws() to concatenate DataFrame multiple columns into a single column. In this article, I will explain the differences between concat() and concat_ws() (concat with…

1 Comment

August 30, 2020