pom.xml for Hadoop 2.6.0

Question

I am trying to implement an app => find maximum temperature from weather data (exepmle from Tom White’s book Hadoop: Definitive Guide (3rd edition)) using Hadoop. I have downloaded and installed Hadoop 2.6.0 what dependencies should I add to make it work?

Expertise

Answer 1

Step by step :

1. Add cloudera your settings.xml (under ${HOME}/.m2/settings.xml) to access hadoop dependencies

   <repository>
        <id>cloudera</id>
        <url>https://repository.cloudera.com/artifactory/cloudera-repos</url>
        <releases>
             <enabled>true</enabled>
        </releases>
        <snapshots>
             <enabled>true</enabled>
        </snapshots>
   </repository>
   

2. Add hadoop dependencies to your pom.xml .

   <dependency>
       <groupId>org.apache.hadoop</groupId>
       <artifactId>hadoop-hdfs</artifactId>
       <version>2.6.0</version>
   </dependency>
   <dependency>
       <groupId>org.apache.hadoop</groupId>
       <artifactId>hadoop-auth</artifactId>
       <version>2.6.0</version>
   </dependency>
   <dependency>
       <groupId>org.apache.hadoop</groupId>
       <artifactId>hadoop-common</artifactId>
       <version>2.6.0</version>
   </dependency>
   <dependency>
       <groupId>org.apache.hadoop</groupId>
       <artifactId>hadoop-core</artifactId>
       <version>1.2.1</version>
   </dependency>
   

3. Then try to "mvn clean install" command into project folder that contains pom.xml file.

Zealot

I always get this error: SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder". SLF4J: Defaulting to no-operation (NOP) logger implementation SLF4J: See slf4j.org/codes.html#StaticLoggerBinder for further details. – Expertise

You should use an implementationof SLF4J , I think this link will help you to configure SLF4J in your project. – Zealot