I created a HBase table from Hive and I'm trying to do a simple aggregation on it. This is my Hive query: from my_hbase_table select col1, count(1) group by col1; The map reduce job spawns onl...

I am trying to run Hadoop balancer command as follows: hadoop balancer -threshold 1 But I am getting several WARN messages as Failed to move blk_1073742036_1212 with size=134217728 from 192.168.30...

I'm trying to find the best approach for logging in MapReduce jobs. I'm using slf4j with log4j appender as in my other Java applications, but since MapReduce job runs in a distributed manner across...

I am trying to implement an app => find maximum temperature from weather data (exepmle from Tom White’s book Hadoop: Definitive Guide (3rd edition)) using Hadoop. I have downloaded and installed Ha...

From the book "Hadoop The Definitive Guide", under the topic Namenodes and Datanodes it is mentioned that: The namenode manages the filesystem namespace. It maintains the filesystem tree and th...

I am new in hadoop and i am not yet familiar to its configuration. I just want to ask the maximum container per node. I am using a single node cluster (6GB ram laptop) and below is my mapred an...

Recently we upgraded to YARN with CDH 5. (Version : 2.3.0 cdh5.1.3, r8e266e052e423af592871e2dfe09d54c03f6a0e8) I was trying to access logs of failed job from Resource Manager by clicking logs...

I am using Hadoop-2.4.0 and my system configs are 24 cores, 96 GB RAM. I am using following configs mapreduce.map.cpu.vcores=1 yarn.nodemanager.resource.cpu-vcores=10 yarn.scheduler.minimum-alloc...

I've instantiated a Hadoop 2.4.1 cluster and I've found that running MapReduce applications will parallelize differently depending on what kind of filesystem the input data is on. Using HDFS, a Ma...

I am reading Hadoop: The definitive guide 3rd edtition by Tom White. It is an excellent resource for understanding the internals of Hadoop, especially Map-Reduce which I am interested in. From the...

After installing and configuring my Hadoop 2.2.0 in pseudo-distributed mode everything is running, as you can see in the jps: $ jps 2287 JobHistoryServer 1926 ResourceManager 2162 NodeManager 1834...