Find out actual disk usage in HDFS

Question

Is there a way to find out how much space is consumed in HDFS? I used

hdfs dfs -df

but it seems to be not relevant cause after deleting huge amount of data with

hdfs dfs -rm -r -skipTrash

the previous comand displays changes not at once but after several minutes (I need up-to-date disk usage info).

Hygroscope

No solution?? Plase comment the answer that show a valid solution. – Worrywart

Answer 1

hadoop cli is deprecated. Use hdfs instead.

Folder wise :

sudo -u hdfs hdfs dfs -du -h /

Cluster wise :

sudo -u hdfs hdfs dfsadmin -report

Odontograph

Answer 2

To see the space consumed by a particular folder try:

hadoop fs -du -s /folder/path

And if you want to see the usage, space consumed, space available, etc. of the whole HDFS:

hadoop dfsadmin -report

Ximenez

Unfortunately hadoop dfsadmin -report shows not up-to-date info too. – Hygroscope

... And the correct command today (2019) is hdfs dfsadmin -report – Worrywart

Answer 3

hadoop fs -count -q /path/to/directory

Unintelligible