When you run the Spark program on windows OS, you often get the exception “Exception in thread “main” java.lang.UnsatisfiedLinkError: org.apache.hadoop.io.nativeio.NativeIO$Windows.access0(Ljava/lang/String;I)Z”
Full-stack trace of the error
Exception in thread “main” java.lang.UnsatisfiedLinkError: org.apache.hadoop.io.nativeio.NativeIO$Windows.access0(Ljava/lang/String;I)Z
Solution: In order to run the Spark program on windows, you would need Hadoop winutils.exe file as windows don’t support HDFS and winutils provides a wrapper.
If you don’t have winutils.exe installed, please download the wintils.exe and hadoop.dll files from https://github.com/steveloughran/winutils (select the Hadoop version you are using as winutils are specific to Hadoop versions)
Copy the downloaded files to a folder on your system, for example, let’s say you have copied these files to c:/hadoop/bin, set the below environment variables.
set HADOOP_HOME=c:/hadoop
set PATH=%PATH%;%HADOOP_HOME%/bin;
Close and reload the command line or terminal to initialize these variables. Sometimes you may also need to put hadoop.dll file into the C:/Windows/System32 folder.
Now run your spark program and issue “Windows.access0(Ljava/lang/String;I)Z” should disappear.
However, if it still doesn’t work, try to restart your system as some of the above settings would get the effect with the restart.
Happy Learning !!