Moving specific files in subdirectories into a directory - python

Question

Im rather new to python but I have been attemping to learn the basics.

Anyways I have several files that once i have extracted from their zip files (painfully slow process btw) produce several hundred subdirectories with 2-3 files in each. Now what I want to do is extract all those files ending with 'dem.tif' and place them in a seperate file (move not copy).

I may have attempted to jump into the deep end here but the code i've written runs without error so it must not be finding the files (that do exist!) as it gives me the else statement. Here is the code i've created

import os

src = 'O:\DATA\ASTER GDEM\Original\North America\UTM Zone 14\USA\Extracted' # input
dst = 'O:\DATA\ASTER GDEM\Original\North America\UTM Zone 14\USA\Analyses' # desired     location

def move():
    for (dirpath, dirs, files) in os.walk(src):
        if files.endswith('dem.tif'):
            shutil.move(os.path.join(src,files),dst)
            print ('Moving ', + files, + ' to ', + dst)
        else:
            print 'No Such File Exists'

Bucella

Answer 1

First, welcome to the community, and python! You might want to change your user name, especially if you frequent here. :)

I suggest the following (stolen from Mr. Beazley):

# genfind.py
#
# A function that generates files that match a given filename pattern

import os
import shutil
import fnmatch

def gen_find(filepat,top):
    for path, dirlist, filelist in os.walk(top):
        for name in fnmatch.filter(filelist,filepat):
            yield os.path.join(path,name)

# Example use

if __name__ == '__main__':
    src = 'O:\DATA\ASTER GDEM\Original\North America\UTM Zone 14\USA\Extracted' # input
    dst = 'O:\DATA\ASTER GDEM\Original\North America\UTM Zone 14\USA\Analyses' # desired     location

    filesToMove = gen_find("*dem.tif",src)
    for name in filesToMove:
        shutil.move(name, dst)

Decorative

Answer 2

Update: the questioner has clarified below that he / she is actually calling the move function, which was the first point in my answer.

There are a few other things to consider:

• You've got the order of elements returned in each tuple from os.walk wrong, I'm afraid - check the documentation for that function.
• Assuming you've fixed that, also bear in mind that you need to iterate over files, and you need to os.join each of those to root, rather than src
• The above would be obvious, hopefully, if you print out the values returned by os.walk and comment out the rest of the code in that loop.
• With code that does potentially destructive operations like moving files, I would always first try some code that just prints out the parameters to shutil.move until you're sure that it's right.

Gagne

Hi Mark, thanks for your reply. Sorry my english is probably not the best what i meant to say is that I am running the program in the shell window by typing in move() and everything runs ok without the shutil.move statement but as soon as i add it i get the following error AttributeError: 'list' object has no attribute 'endswith' – Bucella

@user1041814: That error is because of the first and second points I mentioned above - your files is actually a list of the subdirectories in each directory in the walk. – Gagne

Thanks again for the suggestions! Its sometimes always the simplest of mistakes like getting the order of the os.walk function that gets you! – Bucella

Answer 3

I think you've mixed up the way you should be using os.walk().

for dirpath, dirs, files in os.walk(src):
    print dirpath
    print dirs
    print files
    for filename in files:
        if filename.endswith('dem.tif'):
            shutil.move(...)
        else:
            ...

Juliajulian

Answer 4

Any particular reason you need to do it in Python? Would a simple shell command not be simpler? If you're on a Unix-like system, or have access to Cygwin on Windows:

find src_dir -name "*dem.tif" -exec mv {} dst_dir

Togetherness