I'm trying to install the MS SQL JDBC driver on ubuntu to be used with sqoop for Hadoop. I'm totally new to java and linux, so I'm not sure where to extract everything to.

I opened up localhost:9870 and try to upload a txt file to the hdfs. I see the error message below Failed to retrieve data from /webhdfs/v1/?op=LISTSTATUS: Server Error

I know that both Under-replicated blocks and Mis-replicated blocks occur due to lesser data node count with respect to replication factor set. But what is the difference between them? On re-sett...

I am trying to create partition for my Table inorder to update a value. This is my sample data 1,Anne,Admin,50000,A 2,Gokul,Admin,50000,B 3,Janet,Sales,60000,A I want to update Janet's Departme...

I am trying to run hadoop as a root user, i executed namenode format command hadoop namenode -format when the Hadoop file system is running. After this, when i try to start the name node server, i...

I am trying to run unit test of spark job in windows 7 64 bit. I have HADOOP_HOME=D:/winutils winutils path= D:/winutils/bin/winutils.exe I ran below commands: winutils ls \tmp\hive winutils ...

I have some Apache Parquet file. I know I can execute parquet file.parquet in my shell and view it in terminal. But I would like some GUI tool to view Parquet files in more user-friendly format. Do...

I have access to a hdfs file system and can see parquet files with hadoop fs -ls /user/foo How can I copy those parquet files to my local system and convert them to csv so I can use them? The fi...

How to execute hive commands directly from .hql file?

Currently, I am using the Apache ParquetReader for reading local parquet files, which looks something like this: ParquetReader<GenericData.Record> reader = null; Path path = new Path("userd...

I'm trying to run the spark examples from Eclipse and getting this generic error: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have su...

I am running a Spark application (version 1.6.0) on a Hadoop cluster with Yarn (version 2.6.0) in client mode. I have a piece of code that runs a long computation, and I want to kill it if it takes...

I was wondering between partitioner and combiner, which runs first? I was of the opinion it is the partitiner first and then combiner and then the keys are redirected to different reducers, which ...

I submit my mapreduce jobs from a java application running on windows to the hadoop 2.2 cluster running on ubuntu. In hadoop 1.x this worked as expected but on hadoop 2.2 I get a strange Error: ja...

Hadoop has configuration parameter hadoop.tmp.dir which, as per documentation, is `"A base for other temporary directories." I presume, this path refers to local file system. I set this value to /...

[hadoop-1.0.2] → hadoop jar hadoop-examples-1.0.2.jar wordcount /user/abhinav/input /user/abhinav/output Warning: $HADOOP_HOME is deprecated. ****hdfs://localhost:54310/user/abhinav/input 12/04/15...

When configuring my hadoop namenode for the first time, I know I need to run bin/hadoop namenode -format but running this a second time, after loading data into HDFS, will wipe out everything an...

trying to run MR program version(2.7) in windows 7 64 bit in eclipse while running the above exception occurring . I verified that using 64 bit 1.8 java version and observed that all the hadoop da...

I tried to install hive on a raspberry pi 2. I installed Hive by uncompress zipped Hive package and configure $HADOOP_HOME and $HIVE_HOME manually under hduser user-group I created. When running hi...

I have orc table in hive I want to drop column from this table ALTER TABLE table_name drop col_name; but I am getting the following exception Error occurred executing hive query: OK FAILED: ...

I'm facing a problem running Jobs on an Amazon EMR when I try to write data on S3. This is the stacktrace: org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find any valid local dire...

I install spark on three nodes successfully. I can visit spark web UI and find every worker node and master node is active. I can run the SparkPi example successfully. My cluster info: 10.45.10.3...

As per the definition of "default" attribute in Avro docs: "A default value for this field, used when reading instances that lack this field (optional)." This means that if the corresponding fiel...

Am looking for installing protobuf 2.5.0 on Arch Linux, so that protoc-2.5.0.so is installed on the OS, so that I can go ahead with building hadoop 2.6.0 from source and make my life easy! :) BTW...

I want to convert a .sas7bdat file to a .csv/txt format so that I can upload it into a hive table. I'm receiving the .sas7bdat file from an outside server and do not have SAS on my machine.