Where is the classpath for hadoop set? When I run the below command it gives me the classpath. Where is the classpath set? bin/hadoop classpath I'm using hadoop 2.6.0

I am passing input and output folders as parameters to mapreduce word count program from webpage. Getting below error: HTTP Status 500 - Request processing failed; nested exception is java.la...

I've been getting the following error in several cases: 2017-03-23 11:55:10,794 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Diagnostics report...

Hadoop defintive guide says - Each Namenode runs a lightweight failover controller process whose job it is to monitor its Namenode for failures (using a simple heartbeat mechanism) and ...

I'm currently configuring hadoop on a server running CentOs. When I run start-dfs.sh or stop-dfs.sh, I get the following error: WARN util.NativeCodeLoader: Unable to load native-hadoop library f...

Im new in hadoop and apache environment. Now I'm trying to learn about kafka I've installed hadoop via ambari. And i can see that there are some services installed. Including zookeeper . When i cli...

I would like to know what is the replacement of NULLIF in Hive? I am using COALESCE but its not serving my requirement. My query statement is something like : COALESCE(A,B,C) AS D COALESCE will ...

I am trying to install a single node setup of Hadoop on Ubuntu. I started following the instructions on the Hadoop 2.3 docs. But I seem to be missing something very simple. First, it says to ...

I installed hadoop 2.6.0 in my laptop running Ubuntu 14.04LTS. I successfully started the hadoop daemons by running start-all.sh and I run a WourdCount example successfully, then I tried to run a j...

Hi all I have setup multi node cluster (i.e 5 node) in my network which working fine. now I wanted to insert and retrieve data from cluster using hive thus I have setup hive latest release apache-h...

I am running this command -- sudo -u hdfs hadoop fs -du -h /user | sort -nr and the output is not sorted in terms of gigs, Terabytes,gb I found this command - hdfs dfs -du -s /foo/bar/*tobe...

I am not able to run ALTER TABLE MY_EXTERNAL_TABLE RECOVER PARTITIONS; on hive 1.2, however when i run the alternative MSCK REPAIR TABLE MY_EXTERNAL_TABLE its just listing the partitions which aren...

I have set the replication factor for my file as follows: hadoop fs -D dfs.replication=5 -copyFromLocal file.txt /user/xxxx When a NameNode restarts, it makes sure under-replicated blocks are re...

I am trying to pass a variable (not property) using -D command line option in hadoop like -Dmapred.mapper.mystring=somexyz. I am able to set a conf property in Driver program and read it back in ma...

I am getting this error while performing start-dfs.sh Starting namenodes on [localhost] pdsh@Gaurav: localhost: rcmd: socket: Permission denied Starting datanodes pdsh@Gaurav: localhost: rcmd: s...

want to set mapreduce.job.user.name=myuser Tried spark-submit --class com.MyClass --conf mapreduce.job.user.name=myuser \ --conf spark.mapreduce.job.user.name=myuser \ --master yarn \ --deplo...

I have created a Spark distribution cluster in Linux (Ubuntu) environment to run Spark submit job. I have 3 machines (m1, s1 and s2; m1 is master, s1 and s2 are slaves machines). In master Spark an...

I am new with Hadoop, I have a file to import into hadoop via command line (I access the machine through SSH) How can I import the file in hadoop? How can I check afterward (command)?

I got an exception when i execute the command sudo -u hdfs hdfs balancer -threshold 5. Here is the Exception. RuntimeException: java.lang.IllegalArgumentException: java.net.UnknownHostException:...

I am trying to install Hadoop 2.2.0 in pseudo-distributed mode. While I am trying to start the datanode services it is showing the following error, can anyone please tell how to resolve this? **2*...

My jobs often hang with this kind of message: 14/09/01 00:32:18 INFO spark.MapOutputTrackerMasterActor: Asked to send map output locations for shuffle 0 to spark@*:37619 Would be great if someon...

I am using spark-sql-2.4.1v which is using hadoop-2.6.5.jar version . I need to save my data first on hdfs and move to cassandra later. Hence I am trying to save the data on hdfs as below: String ...

I have installed Java openjdk version "10.0.2"and Hadoop 2.9.0 successfully. All processes are running well hadoopusr@amalendu:~$ jps 19888 NameNode 20388 DataNode 20898 NodeManager 20343 Seconda...

I'm building a spring-boot powered service that writes data to Hadoop using filesystem API. Some data is written to parquet file and large blocks are cached in memory so when the service is shut do...

I have set up a multi node Hadoop Cluster. The NameNode and Secondary namenode runs on the same machine and the cluster has only one Datanode. All the nodes are configured on Amazon EC2 machines. ...