Hadoop “WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform” warning
How to resolve Hadoop warning "WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform……
3 Comments
October 19, 2020
Photo by Christina Winter on Unsplash
Spark Read Files from HDFS (TXT, CSV, AVRO, PARQUET, JSON)
Though Spark supports to read from/write to files on multiple file systems like Amazon S3,…
March 21, 2020
Spark Step-by-Step Setup on Hadoop Yarn Cluster
This post explains how to setup Apache Spark and run Spark applications on the Hadoop…
January 10, 2019
Hadoop Yarn Configuration on Cluster
This post explains how to setup Yarn master on hadoop 3.1 cluster and run a map reduce program. Before you proceed this document, please make sure you have Hadoop3.1 cluster up and running. if you do not have a setup, please follow below link to setup your cluster and come back to this page.
January 10, 2019
How Yarn works
YARN / Map Reduce 2 (Yet Another Resource Negotiator) Resource Manager The ResourceManager is…
December 17, 2018
WARNING: “HADOOP_PREFIX has been replaced by HADOOP_HOME. Using value of HADOOP_PREFIX.
Are you getting WARNING: "HADOOP_PREFIX has been replaced by HADOOP_HOME. Using value of HADOOP_PREFIX. ? for every command you issue on a cluster, follow below step to resolve it.
December 9, 2018
Incompatible clusterIDs error in datanode logs
When your datanodes are not starting due to java.io.IOException: Incompatible clusterIDs error, means you have formatted namenode with out deleting files from datanode.
December 9, 2018
Apache Hadoop Installation on Ubuntu (multi-node cluster).
Below are the steps of Apache Hadoop Installation on a Linux Ubuntu server, if you…
December 8, 2018