Create missing directories in ftplib storbinary

Question

I was using pycurl to transfer files over ftp in python. I could create the missing directories automatically on my remote server using:

c.setopt(pycurl.FTP_CREATE_MISSING_DIRS, 1)

for some reasons, I have to switch to ftplib. But I don't know how to to the same here. Is there any option to add to storbinary function to do that? or I have to create the directories manually?

Hereditable

Answer 1

FTP_CREATE_MISSING_DIRS is a curl operation (added here). I'd hazard a guess that you have to do it manually with ftplib, but I'd love to be proven wrong, anyone?

I'd do something like the following: (untested, and need to catch ftplib.all_errors)

ftp = ... # Create connection

# Change directories - create if it doesn't exist
def chdir(dir): 
    if directory_exists(dir) is False: # (or negate, whatever you prefer for readability)
        ftp.mkd(dir)
    ftp.cwd(dir)

# Check if directory exists (in current location)
def directory_exists(dir):
    filelist = []
    ftp.retrlines('LIST',filelist.append)
    for f in filelist:
        if f.split()[-1] == dir and f.upper().startswith('D'):
            return True
    return False

Or you could do directory_exists like this: (a bit harder to read?)

# Check if directory exists (in current location)
def directory_exists(dir):
    filelist = []
    ftp.retrlines('LIST',filelist.append)
    return any(f.split()[-1] == dir and f.upper().startswith('D') for f in filelist)

Valer

Thank you, although it was not exactly what I was looking for, but it was a good answer. Thanx ;) – Hereditable

No, you don't have to do it manually. You could just call the makedirs method in the ftputil package instead. – Liles

Note this solution only works with top level directories and does not work with sub-directories – Skinflint

Answer 2

I know it's kind of an old post but I just needed this and came up with a very simple function. I'm new to Python so I'd appreciate any feedback.

from ftplib import FTP

ftp = FTP('domain.com', 'username', 'password')

def cdTree(currentDir):
    if currentDir != "":
        try:
            ftp.cwd(currentDir)
        except IOError:
            cdTree("/".join(currentDir.split("/")[:-1]))
            ftp.mkd(currentDir)
            ftp.cwd(currentDir)

Usage example:

cdTree("/this/is/an/example")

Wozniak

very nice ! dir is a python built-in you might want to change that variable name... also you want to catch specific exceptions, not all of them – Liles

Thanks you xApple for your feedback. I replaced 'dir' and restricted to only catch IOError exceptions. – Wozniak

I think you forgot to replace an instance of the dir variable. – Liles

Oops, fixed it. Thank you for catching it. Now it's perfect, it just needs to be up voted. :) – Wozniak

@Wozniak - a couple more suggestions: lower_case_with_underscores is preferred for function/variable names - see pep8. Also it's better to use os.path functions for manipulating paths, e.g. os.path.normpath(os.path.join(current_dir, '..')) – Valer

smart solution. Exactly what i was looking for.! – Dense

I'd like to highlight that this also answers another common question: how do I check if a ftp directory exists. I think this way is more Pythonic than the answer of listing the directory contents then looking for the directory name. Instead, try to change to that directory, and catch the error, in line with "ask for forgiveness" rather than "ask for permission". – Adrianaadriane

In my opinion try and except is very pythonic. This question deals with it: stackoverflow.com/questions/7604636 – Mundford

This no longer works (with me). I had to change ioerror to ftplib.error_perm, and ftp.mkd(currentDir) to ftp.mkd(currentDir.split("/")[-1]). – Forestforestage

even in 2022 this is the best way of doing it – Tacnode

Answer 3

I tried adding this as a comment to the @Alex L 's answer, but it was too long. You need to descend recursively when changing directory if you want to create directories on the way. E.g.

def chdir(ftp, directory):
    ch_dir_rec(ftp,directory.split('/')) 

# Check if directory exists (in current location)
def directory_exists(ftp, directory):
    filelist = []
    ftp.retrlines('LIST',filelist.append)
    for f in filelist:
        if f.split()[-1] == directory and f.upper().startswith('D'):
            return True
    return False

def ch_dir_rec(ftp, descending_path_split):
    if len(descending_path_split) == 0:
        return

    next_level_directory = descending_path_split.pop(0)

    if not directory_exists(ftp,next_level_directory):
        ftp.mkd(next_level_directory)
    ftp.cwd(next_level_directory)
    ch_dir_rec(ftp,descending_path_split)

Adjacency

Answer 4

I am using the following lines to resolve missing directory paths for FTP file copy

import os
ftps = FTP_TLS('ftps_server')
ftps.connect()
ftps.login()

destination_dir_path = 'some/dir/path'          # directory path on FTP
dir_path = ''
for i in destination_dir_path.split('/'):
    dir_path = os.path.join(dir_path,i)
    if i not in ftps.nlst(os.path.dirname(dir_path)):
        ftps.mkd(dir_path)                      # create directory on the FTP
ftps.storbinary(...)                            # store file using the binary mode

Marquettamarquette

Answer 5

A more robust and reliable solution:

using ftplib

Hostname = yourhostname.com
Username = yourusername
Password = yourpassword

def mkdirs(path):
    ftp = FTP(Hostname,Username,Password) 
    items = path.split('/')
    cwd = "/"
    for i in range(len(items)):
        list = ftp.nlst()
        if(not '.' in items[i] and not items[i] in list):
            ftp.mkd(cwd + items[i] + "/")
            cwd += items[i] + '/'
            ftp.cwd(cwd)
    ftp.quit()

mkdirs('path/to/directory/file.name')

This will create directories on your server if they do not exist.

Limitations: This will not work on folders with names that contain ..

Terzas

I edited slightly, as it errors when a dir – Jellicoe

Answer 6

An alternative is to simply loop through each of the path elements, create the next and change into the newly-created directory. My use case was fairly straightforward though as I was copying items from one FTP server to another.

def create_ftp_path(session: ftplib.FTP, required_dir: str):
    required_dir = required_dir.split('/')[:-1]
    for path_item in required_dir:
        if path_item.strip() == '':
            continue
        path_item = path_item.replace('/', '')
        try:
            session.cwd(path_item)
        except:
            session.mkd(path_item)
            session.cwd(path_item)

Considerations:

• This function assumes you have already changed directory for your FTP session to some base path and the required_dir is a path from that base path.
• required_dir includes file name as the last element.
• I'm removing any / characters because in my case they were causing 553 permission denied exception.
• The exception handling is lacking, but in my case upload validation is happening further in the code so even if it fails it will be caught further down.

Aspirate

Answer 7

This code will create all missing folders in path:

...

def chdir(ftp_path, ftp_conn):
    dirs = [d for d in ftp_path.split('/') if d != '']
    for p in dirs:
        print(p)
        check_dir(p, ftp_conn)


def check_dir(dir, ftp_conn):
    filelist = []
    ftp_conn.retrlines('LIST', filelist.append)
    found = False

    for f in filelist:
        if f.split()[-1] == dir and f.lower().startswith('d'):
            found = True

    if not found:
        ftp_conn.mkd(dir)
    ftp_conn.cwd(dir)

if __name__ == '__main__':
    ftp_conn = ... # ftp connection
    t = 'FTP/for_Vadim/1/2/3/'

    chdir(t, ftp_conn)

This code will check all dirs in path and create missing dirs

before "FTP/for_Vadim/" after "FTP/for_Vadim/1/2/3/"

Miguelmiguela

Answer 8

I'm using something like this (without cwd):

# -*- coding:utf-8 -*-

from ftplib import FTP, error_perm


def createDirs(ftp, dirpath):
    """
    Create dir with subdirs.

    :param ftp:     connected FTP
    :param dirpath: path (like 'test/test1/test2')

    :type ftp:      FTP
    :type dirpath:  str
    :rtype:         None

    """

    dirpath = dirpath.replace('\\', '/')
    tmp = dirpath.split('/')
    dirs = []

    for _ in tmp:
        if len(dirs) == 0:
            dirs.append(_)
            continue

        dirs.append(dirs[-1] + '/' + _)

    for _ in dirs:
        try:
            ftp.mkd(_)
        except error_perm as e:
            e_str = str(e)
            if '550' in e_str and 'File exists' in e_str:
                continue


if __name__ == '__main__':
    # init ftp
    createDirs(ftp=ftp, dirpath='test/1/2/3')

Huskamp