Sqoop Import --password-file function not working properly in sqoop 1.4.4

Question

I am using hadoop-1.2.1 and sqoop version is 1.4.4.

I am trying to run the following query.

sqoop import --connect jdbc:mysql://IP:3306/database_name --table clients --target-dir /data/clients --username root --password-file /sqoop.password -m 1

sqoop.password is a file which is kept on HDFS in path /sqoop.password with permission 400.

It is giving me an error

Access denied for user 'root'@'IP' (using password: YES)

Can anyone provide solution for this? Thanks in advance.

Beaconsfield

Answer 1

"\n" is being written in file when you vi the file and write the password. Better use the below approach to avoid problems

echo -n "Your_sqoop_password" > sqoop.password

Helmand

Exactly. the password should be naked, no newline. – Seppala

Or use vim as per #1051140, since part of the point of using a password file is to avoid the password appearing wherever command line usages appear in logs or ps. – Cataract

Perfect, this fixed my issue. – Tambac

Answer 2

Not sure if you are still having this issue. The password file can be in any folder. Try the following syntax and it should work:

--password-file file:///user/root/database.password

Resurrectionist

Thanks a lot. I had the same problem and got solved with this answer. – Supinate

Thanks, and it worked for me as well. I would add that if you want it to look in hdfs, for me I had to use the following: --password-file hdfs:///user/root/database.password – Knickknack

I was getting an error that the provided password file did not exist. But adding file:/// as per this answer helped me solve that error – Nugent

Answer 3

As per the sqoop documentation

You should save the password in a file on the users home directory with 400 permissions and specify the path to that file using the --password-file argument, and is the preferred method of entering credentials. Sqoop will then read the password from the file and pass it to the MapReduce cluster using secure means with out exposing the password in the job configuration. The file containing the password can either be on the Local FS or HDFS.

If I am running my sqoop job with root user then my password file will be in /user/root/ in HDFS

sqoop import --connect jdbc:mysql://database.example.com/employees \
    --username venkatesh --password-file /user/root/database.password

For more details you can check this

Mabe

Hi Prasad, Didn't work on my end. It is giving the same error. – Beaconsfield

I had to place the password file into HDFS to get it to work. sigh. – Seppala

yes as per the documentation that I have added above The file containing the password can either be on the Local FS or HDFS. and im my case I have placed the password file in HDFS – Mabe

Answer 4

While creating password, use echo -n option. (-n option removes all trailing spaces).
Suppose you have a password "myPassword" and you want to save it to a file sqoop.password, then follow below steps:

1. Create password using command

   echo -n "myPassword" > sqoop.password
   

2. Upload the file to HDFS as the file needs to be present in HDFS

   hadoop fs -put sqoop.password /user/keepMyFilesHere
   

3. Write the scoop import command

   sqoop list-tables --connect jdbc:mysql://localhost/kpdatabase --username root --password-file /user/karanpreet.singh/sqoop.password
   

This will definitely work!

Angeliaangelic

The subtlety that the password file must be in hadoop is missing from other answers, so this answer is more complete! – Seppala

Using echo is not advised for shared environments since it makes the password visible to other users. – Cataract

Answer 5

Check if there is any garbage character in your password file. I was getting the same issue and finally found that the file contained a \n character at the end and sqoop considered that also as a part of the password string. Try creating a password file as mentioned below and than use the password file : echo -n "root_password" > password.txt Place your password in place of root_password.

Libre

On linux, ls -l, or wc will disclose the file size, and od -c or od -c (do you prefer character or hex output?) will show you the contents. – Seppala

Answer 6

1. Verify that the user running the sqoop command is the owner of the file /sqoop.password (which has the 400 permissions set). This will ensure that the password file is readable.

2. Make sure that the sqoop.password file does not have any extra non-printing characters at the end (for example, a newline character). One easy way to check this is to look at the size of the file. If the password is say 'sqoop', the size of the file should be 5. If it is larger, delete the sqoop.password file from HDFS, regenerate it locally and put it back on HDFS.
   
   The command that should report the above information pieces is:
   
   hadoop fs -ls /sqoop.password

Shaughn

Answer 7

please follow below link for all your sqoop password related doubts.

www.ericlin.me

Peneus