Python os.path.join on Windows

Question

I am trying to learn python and am making a program that will output a script. I want to use os.path.join, but am pretty confused. According to the docs if I say:

os.path.join('c:', 'sourcedir')

I get "C:sourcedir". According to the docs, this is normal, right?

But when I use the copytree command, Python will output it the desired way, for example:

import shutil
src = os.path.join('c:', 'src')
dst = os.path.join('c:', 'dst')
shutil.copytree(src, dst)

Here is the error code I get:

WindowsError: [Error 3] The system cannot find the path specified: 'C:src/*.*'

If I wrap the os.path.join with os.path.normpath I get the same error.

If this os.path.join can't be used this way, then I am confused as to its purpose.

According to the pages suggested by Stack Overflow, slashes should not be used in join—that is correct, I assume?

Molar

Answer 1

Windows has a concept of current directory for each drive. Because of that, "c:sourcedir" means "sourcedir" inside the current C: directory, and you'll need to specify an absolute directory.

Any of these should work and give the same result, but I don't have a Windows VM fired up at the moment to double check:

"c:/sourcedir"
os.path.join("/", "c:", "sourcedir")
os.path.join("c:/", "sourcedir")

Buckjump

os.path.join('C:/', 'sourcedir') worked as expected. I thank you very much good sir :) the others '//' 'c:' 'c:\\' did not work (C:\\ created two backslashes, C:\ didn't work at all) Thanks again ghostdog74, Smashery, and Roger Pate. I am in your debt :) – Molar

Sorry, line breaks weren't kept in comment, it looks very messy – Molar

Even if this works in some cases, the answer by @Pumpkinseed is a much better solution. Using os.sep will choose between / and \ depending on OS. – Nikolos

Is there any point in using os.path.join or os.sep if you're going to specify c: anyway? c: makes no sense on other OSs. – Chromato

all these solutions are only partially satisfying. It is ok to manually add the separator when you have a single specific case, but in case you want to do it programmatically, what is the criteria for which os.path.join('c:','folder') works differently from os.path.join('folder','file')? Is it because of the : or because 'c:` is a drive? – Torrent

the rule is probably 'don't add sep if single letter follow by colon'. I have implemented this rule in an answer below. – Torrent

Answer 2

To be even more pedantic, the most python doc consistent answer would be:

mypath = os.path.join('c:', os.sep, 'sourcedir')

Since you also need os.sep for the posix root path:

mypath = os.path.join(os.sep, 'usr', 'lib')

Pumpkinseed

Excuse my ignorance - It looks like the code still varies between Windows and Linux, so what makes os.sep superior? – Manumission

Please note this snafu when trying to inject os.sep. It only works after the bare drive letter. >>> os.path.join("C:\goodbye", os.sep, "temp") 'C:\\temp' – Wheelwright

@Manumission my answer builds off of this one to provide a system-agnostic solution: https://mcmap.net/q/179624/-python-os-path-join-on-windows – Gounod

I don't understand the point of all these "pedantic" solutions. os.sep is useful when you want to manipulate paths without making assumptions about the separator. It's pointless to use with os.path.join() since it already knows the right separator. It's also pointless if you end up needing to explicitly specify the root directory by name (as you can see in your own example). Why do "c:" + os.sep instead of simply "c:\\", or os.sep + "usr" instead of simply "/usr"? Also note that in Win shells you can't cd c: but you can cd c:\ , suggesting that the root name is actually c:\ . – Myrnamyrobalan

Point taken about the use of "\" and "/" for practical purposes. But, I don't know what point was trying to be made about "can't CD C:". In a Windows CMD shell "CD C:" shows the present CD value of the C: drive. You can certainly do this. The command "CD /D C:" will change the current directory to the present CD value of the C: drive. This is useful when you're on drive F: and want to move to drive C:. But it is easier to just type 'C:' instead of 'CD /D C:'. – Tribromoethanol

Answer 3

To be pedantic, it's probably not good to hardcode either / or \ as the path separator. Maybe this would be best?

mypath = os.path.join('c:%s' % os.sep, 'sourcedir')

or

mypath = os.path.join('c:' + os.sep, 'sourcedir')

Fermium

Answer 4

For a system-agnostic solution that works on both Windows and Linux, no matter what the input path, one could use

def joinpath(rootdir, targetdir):
    return os.path.join(os.sep, rootdir + os.sep, targetdir)

On Windows:

>>> joinpath("C:", "Windows")
'C:\\Windows'
>>> joinpath("C:\\", "Windows")
'C:\\Windows'
>>> joinpath("C:\\Windows", "src")
'C:\\Windows\\src'

On Linux:

>>> joinpath("usr", "lib")
'/usr/lib'
>>> joinpath("/usr", "lib")
'/usr/lib'
>>> joinpath("/usr/lib", "src")
'/usr/lib/src'

Gounod

Thanks! This is even more useful since it doesn't suffer from the gotcha that @Wheelwright mentioned earlier: os.path.join(os.sep, "C:\\a" + os.sep, "b") returns "C:\\a\\b" on Windows. – Manumission

How are either of these examples system agnostic though? c: doesn't exist on *nix, and usr doesn't exist on windows.. – Chromato

The function call os.path.join(os.sep, rootdir + os.sep, targetdir) is system agnostic precisely because it works with both of those system-specific examples, without needing to change the code. – Gounod

This solution, much like the earlier post that inspired it, still relies on setting rootdir like rootdir = "usr" if nix else "c:". But the more direct and accurate rootdir = "/usr" if nix else "c:\\" works just as well, without the os.sep acrobatics and ensuing head scratching. There's no danger that a root directory on *nix will start with anything other than a forward slash, or that Windows will have root directories named without a trailing colon and backslash (e.g. in Win shells, you can't just do cd c:, you'd need to specify the trailing backslash), so why pretend otherwise? – Myrnamyrobalan

Answer 5

The reason os.path.join('C:', 'src') is not working as you expect is because of something in the documentation that you linked to:

Note that on Windows, since there is a current directory for each drive, os.path.join("c:", "foo") represents a path relative to the current directory on drive C: (c:foo), not c:\foo.

As ghostdog said, you probably want mypath=os.path.join('c:\\', 'sourcedir')

Gerstein

Answer 6

I'd say this is a (windows)python bug.

Why bug?

I think this statement should be True

os.path.join(*os.path.dirname(os.path.abspath(__file__)).split(os.path.sep))==os.path.dirname(os.path.abspath(__file__))

But it is False on windows machines.

Nosology

I'm inclined to agree that that constitutes a Python bug. Is this still the case? (Written from the glorious utopian future of late 2015.) – Blas

I cannot answer this question with respect to windows, since I do not have access to a windows machine, but I guess python's behavior regarding this question hasn't changed. Anyway, this statement is also not true for Linux implementations, since the first statement returns the path without the leading separator (a.k.a the root directory), whereas the second statement returns the path including the leading separator. – Nosology

So I actually do not like my answer regarding this question anymore. But I also don't like python's behavior regarding this. – Nosology

@Cecil I am on this question right now because of the same issue... it does appear to still be the case. – Coating

Answer 7

to join a windows path, try

mypath=os.path.join('c:\\', 'sourcedir')

basically, you will need to escape the slash

Pharynx

Answer 8

You have a few possible approaches to treat path on Windows, from the most hardcoded ones (as using raw string literals or escaping backslashes) to the least ones. Here follows a few examples that will work as expected. Use what better fits your needs.

In[1]: from os.path import join, isdir

In[2]: from os import sep

In[3]: isdir(join("c:", "\\", "Users"))
Out[3]: True

In[4]: isdir(join("c:", "/", "Users"))
Out[4]: True

In[5]: isdir(join("c:", sep, "Users"))
Out[5]: True

Munich

Answer 9

Consent with @georg-

I would say then why we need lame os.path.join- better to use str.join or unicode.join e.g.

sys.path.append('{0}'.join(os.path.dirname(__file__).split(os.path.sep)[0:-1]).format(os.path.sep))

Clavicembalo

yeah, right, it's waaaaayy clearer that way. Why not using regexes while you're at it? or call a perl script and process the output? – Laszlo

I don't think it's a good idea because os.path.join is pretty good semantics... So you see it in a code and understand straight away what is going on. – Nikolos

Answer 10

answering to your comment : "the others '//' 'c:', 'c:\\' did not work (C:\\ created two backslashes, C:\ didn't work at all)"

On windows using os.path.join('c:', 'sourcedir') will automatically add two backslashes \\ in front of sourcedir.

To resolve the path, as python works on windows also with forward slashes -> '/', simply add .replace('\\','/') with os.path.join as below:-

os.path.join('c:\\', 'sourcedir').replace('\\','/')

e.g: os.path.join('c:\\', 'temp').replace('\\','/')

output : 'C:/temp'

Vitrics

Answer 11

The proposed solutions are interesting and offer a good reference, however they are only partially satisfying. It is ok to manually add the separator when you have a single specific case or you know the format of the input string, but there can be cases where you want to do it programmatically on generic inputs.

With a bit of experimenting, I believe the criteria is that the path delimiter is not added if the first segment is a drive letter, meaning a single letter followed by a colon, no matter if it corresponds to a real unit.

For example:

import os
testval = ['c:','c:\\','d:','j:','jr:','data:']

for t in testval:
    print ('test value: ',t,', join to "folder"',os.path.join(t,'folder'))

test value:  c: , join to "folder" c:folder
test value:  c:\ , join to "folder" c:\folder
test value:  d: , join to "folder" d:folder
test value:  j: , join to "folder" j:folder
test value:  jr: , join to "folder" jr:\folder
test value:  data: , join to "folder" data:\folder

A convenient way to test for the criteria and apply a path correction can be to use os.path.splitdrive comparing the first returned element to the test value, like t+os.path.sep if os.path.splitdrive(t)[0]==t else t.

Test:

for t in testval:
    corrected = t+os.path.sep if os.path.splitdrive(t)[0]==t else t
    print ('original: %s\tcorrected: %s'%(t,corrected),' join corrected->',os.path.join(corrected,'folder'))

original: c:    corrected: c:\  join corrected-> c:\folder
original: c:\   corrected: c:\  join corrected-> c:\folder
original: d:    corrected: d:\  join corrected-> d:\folder
original: j:    corrected: j:\  join corrected-> j:\folder
original: jr:   corrected: jr:  join corrected-> jr:\folder
original: data: corrected: data:  join corrected-> data:\folder

it can be probably be improved to be more robust for trailing spaces, and I have tested it only on windows, but I hope it gives an idea. See also Os.path : can you explain this behavior? for interesting details on systems other then windows.

Torrent

Answer 12

I got around this by using:

os.sep.join(list('C:', 'sourcedir'))

join here is not from os.path.join(), but from ''.join()

It can be useful in the following situation:

import os

some_path = r'C:\some_folder\some_file.txt'
path_items = some_path.split(os.sep)
same_path = os.sep.join(path_items)

Conserve

Answer 13

For simplicity's sake, a workaround:

my_path = r'D:\test\test2\file.txt' # original path string
drive_letter = my_path.split(':')[0] # avoid os.path.join relative path bs
my_path_parts = os.path.normpath(my_path.split(':')[1]).split(os.sep)

# do some path cmponent modifications here if you want

if drive_letter: # if drive letter exists
    drive_letter += ':\\'

my_new_path = drive_letter + os.path.join(*my_path_parts)
my_new_path

Samalla

Answer 14

In windows using os.paht.join("/", "Temp") will result in /Temp by default but as strange as it sounds, there is no problem in using that path as a full path equivalent to "C:/Temp" and it works for both saving files and opening files.

Outpouring