Is there any option in sqoop to import data from RDMS and store it as ORC file format in HDFS? Alternatives tried: imported as text format and used a temp table to read input as text file and writ...

I could not understand the difference between the following commands in sqoop. It would be better if someone could explain with small examples. --warehouse-dir and --target-dir Thanks

Both Flume and Sqoop are meant for data movement, then what is the difference between them? Under what condition should I use Flume or Sqoop?

I have a 6 node cloudera based hadoop cluster and I'm trying to connect to an oracle database from a sqoop action in oozie. I have copied my ojdbc6.jar into the sqoop lib location (which for me ha...

hduser@kubuntu17:~$ sqoop version /usr/local/hadoop/libexec/hadoop-functions.sh: line 2326: HADOOP_ORG.APACHE.SQOOP.SQOOP_USER: bad substitution /usr/local/hadoop/libexec/hadoop-functions.sh: line ...

I am using sqoop 1.4.2 version. I am trying to change the sqoop metastore from default hsqldb to mysql. I have configured following properties in sqoop-site.xml file. <property> <name&...

I'm trying to Sqoop the data from Teradata to hive. I thought of following the below steps: 1) Create a Hive table with all the required fields in Hue. 2) By using Sqoop import command along with ...

I am using the version of Apache Oozie 4.3.0 along with Hadoop 2.7.3 I have developed a very simple Oozie workflow, which simply has a sqoop action to export system events to a MySQL table. <w...

Is it possible to overwrite HDFS directory automatically instead of overwriting it every time manually while Sqoop import? (Do we have any option like "--overwrite" like we have for hive import "-...

We have implemented a solution using Sqoop to load data from RDBMS to our hadoop cluster, for append-only data, it goes to hive while dimension data to hbase. Now we are setting up two identical ...

Just wondering can sqoop run without a hadoop cluster? sort of in a standalone mode? Has anyone tried to run sqoop on spark, please share some experiences on it.

I am using Sqoop (version 1.4.4) to import data from MySQL to Hive. The data will be a subset of one of tables, i.e. few columns from a table. Is it necessary to create table in Hive before hand. O...

It is possible to export data from HDFS to RDBMS table using Sqoop. But it seems like we need to have existing table. Is there some parameter to tell Sqoop do the 'CREATE TABLE' thing and export d...

Sqoop able to import data from multiple tables using --query clause but not clear whether it is able to import below query. Select deptid, avg(salary) from emp group by deptid Another question...

I wrote a simple script to create a user (TestV100), create a table (Xy100) in that schema and export a tabl delimited flat file from hadoop to this oracle table. This is the shell script: - Expor...

In our organization, we have been trying to use hadoop ecosystem based tools to implement ETLs lately. Although the ecosystem itself is quite big, we are using only a very limited set of tools at t...

I'm crceating a sqoop job which will be scheduled in Oozie to load daily data into Hive. I want to do incremental load into hive based on Date as a parameter, which will be passed to sqoop job Af...

I am trying to import a table from Netezza DB using sqoop hcatlog ( see below) in ORC format as suggested here Sqoop command: sqoop import -m 1 --connect <jdbc_url> --driver <database_dr...

Using Sqoop to import data from oracle to hive, its working fine but it create table in hive with only 2 dataTypes String and Double. I want to use timeStamp as datatype for some columns. How can I...

sqoop import --connect jdbc:mysql://remote-ip/db --username xxx --password xxx --table tb --hive-import The above command imports table tb into the 'default' Hive database. Can I use other datab...

On HDP 2.3.2 with Sqoop 1.4.6, I'm trying to import tables from SQL Server 2008. I'm able to successfully connect to the SQL Server because I can list databases and tables etc. However, every sin...

So with mapreduce v2 you can use binding to certain YARN queues to manage resources and prioritization. Basically by using "hadoop jar /xyz.jar -D mapreduce.job.queuename=QUEUE1 /input /output" w...

Can anyone tell the difference between create-hive-table & hive-import method? Both will create a hive table, but still what is the significance of each?

I have a hive query: insert override directory /x select ... Then I'm try to export the data with sqoop sqoop export --connect jdbc:mysql://mysqlm/site --username site --password site --table ...

When I execute: sqoop import --connect jdbc:mysql://localhost/testdb --table test --hive-table test --hive-import -m 1 I get following error message: 13/04/21 16:42:50 ERROR tool.ImportTool: E...