How to clone files with Python?

Asked 22/12, 2017 at 17:54 Answered 22/12, 2017 at 18:34

Using bash on macos I can create COW file clones with cp -c. Is there a Python library that provides the same functionality? The copy functions in shutil doesn't seem to mention cloning.

On APFS clones: https://developer.apple.com/library/content/documentation/FileManagement/Conceptual/APFS_Guide/Features/Features.html

On BSD clonefile: http://www.manpagez.com/man/2/clonefile/

Biforked answered 22/12, 2017 at 17:54 Comment(8)

Is that a feature of the filesystem? – Bauske 22/12, 2017 at 18:0

#123698 – Vedi 22/12, 2017 at 18:2

By any chance, are you looking for a cli solution? – Melano 22/12, 2017 at 18:7

do you just need a cow buffer? in python this is typically referred to as mmap I haven't been able to find any information on cow files on osx... – Skeg 22/12, 2017 at 18:9

I take it back, I just found an article talking about APFS adding cow (among other things...) – Skeg 22/12, 2017 at 18:11

python does not seem to support the bsd copyfile command (which is what cp -c relies on), so you'll have to make system calls as Dylan suggested. – Skeg 22/12, 2017 at 18:21

@DylanB This is clearly not a duplicate of the questions you have linked to, would you mind undoing marking this is as a duplicate. I feel like it was pretty clear in the question that I was referring to COW file clones. – Biforked 22/12, 2017 at 18:27

Flag retracted. – Vedi 22/12, 2017 at 18:28

The Python standard library does not support cloning files.

To clone a file using cp -c in a subprocess you can use this function:

def clonefile(source, dest):
    subprocess.check_output(["cp", "-c", source, dest])

Biforked answered 22/12, 2017 at 18:34 Comment(0)

-1

Ok, I marked this as a duplicate but didn't know the difference between copying and cloning.

This is the location of the instructions for copying files with as much detail as possible in Python: https://docs.python.org/3.6/library/shutil.html

It starts with the following warning: Warning Even the higher-level file copying functions (shutil.copy(), shutil.copy2()) cannot copy all file metadata.

If what you want is an OS cloning of the file, you're going to need to run a system call. That system call is going to need to be OS specific, and the call will be pulled from this question: Calling an external command in Python

So, look into the copy library and see if you can get enough detail, and if you can't use a system-specific call.

Vedi answered 22/12, 2017 at 18:7 Comment(3)

your reference to external commands may be useful, but the OP already covered that shutil is not an acceptable solution. – Skeg 22/12, 2017 at 18:16

All the documentation I found pointed to shutil being as deep as Python goes.... I was a bit trigger happy on the answer, I admit. – Vedi 22/12, 2017 at 18:21

And, bluntly, it wouldn't make much sense for Python to have standard libraries system-specific enough to keep up with the power of Bash, ksh, etc. A compiled solution would need to do all sorts of system lookups and (potentially bug ridden) dancing to keep up. So I'd say you're better off sticking with the tool that does what you specifically need. – Vedi 22/12, 2017 at 18:27

Recommended topics

Hot tags