copyFromLocal: unexpected URISyntaxException

Question

I am trying to copy some files from my hard drive to HDFS , I am using this command

 hadoop fs -copyFromLocal /home/hduser/Pictures/Event\ ordering/* input/

Is this the correct syntax ?

PS : I am using linux

PPS: I get this error

      copyFromLocal: unexpected URISyntaxException

Greensickness

It seems the whitespace, eventhough escaped, gets evaluated twice. Try escaping it twice, that is hadoop fs -copyFromLocal /home/hduser/Pictures/Event\\\ ordering/* input/. – Yahrzeit

@Yahrzeit I don't think it the problem that error occurred twice cause I have 2 field in that directory. – Greensickness

Do your filenames contain colon :? It is not supported at the moment issues.apache.org/jira/browse/HADOOP-7945 – Yahrzeit

@Yahrzeit unfortunately yeah ! thanks a lot ! how can I mark you comment as an answer? – Greensickness

@Yahrzeit I spoke too early , I tried this on a file without ":" hadoop fs -copyFromLocal /home/hduser/Pictures/openCV+Tank\ Detection/10.png input still the same error. – Greensickness

Hadoop URI Syntax follows web URI syntax. Spaces must be represented as %20. Please try hadoop fs -copyFromLocal /home/hduser/Pictures/Event%20ordering/* input/. But this will cause the shell not to expand * into filenames and you'll get files not found. Try hadoop fs -copyFromLocal /home/hduser/Pictures/openCV+Tank%20Detection/10.png input and it should work. – Yahrzeit

@Yahrzeit hadoop fs -copyFromLocal /home/hduser/Pictures/openCV+Tank%20Detection/10.png input worked , Any Idea how do I get to transfer all the files ? These guys just keep getting worse !! – Greensickness

Let us continue this discussion in chat. – Greensickness

The easiest is to build the list of files then run copyFromLocal. Try this for file in /home/hduser/Pictures/Event\ ordering/*; do files=${files},${file// /%20}; done; hadoop fs -copyFromLocal $(eval echo {${files#,}}) input. – Yahrzeit

@Yahrzeit can i contact you via mail, i have one question regarding hadoop namenode? – Steffin

@Steffin - Sure. You'll find my email address in my profile. – Yahrzeit

Answer 1

You need to represent Space as %20, i.e.

Event ordering -> Event%20ordering

try

hadoop fs -copyFromLocal /my/local/dir/Event%20ordering/* input/

It will work.

Interpret

Answer 2

Remove space between Directory name and it will work

steps are as follows

1.Rename directory name and remove space between them

change Event ordering to Eventordering

2 now run following command

hadoop fs -copyFromLocal /home/hduser/Pictures/Eventordering/* input/

Handfasting

Answer 3

This happens when you have files with space. Linux doesn't recognize spaces in their filenames. So, if you want to move those files, you first have to remove those spaces and then execute. Also, it is always recommended to not have space in Linux.

hadoop fs -copyFromLocal /home/hduser/Pictures/Eventordering/* input/

Tightwad