How to check for hidden files & folders on NTFS partition using python on linux?

Question

I am using NTFS partition on linux machine. I want to identify hidden files and folders on my NTFS partition on linux using python.

How can I achieve this using python. Any code snippet / links would be appreciated.

Thanks.

Spleeny

What have you got so far? Can you identify non-hidden files? – Caldwell

@Johnsyweb: I just have the code to identify the hidden files on linux platform.. i.e. all files / folders starting with "." (dot character). My requirement is to detect the hidden files and folders on a USB harddisk which has NTFS partition and get connected to my linux machine.. – Spleeny

Do you mean files with the "hidden" attribute? Or some sort of special files not shown by os.walk? – Fastening

@AndiDog: Yes, I mean files with "hidden" attribute. I have problem with detecting hidden files & folders of Windows files which are accessed by connecting the USB hard disk having NTFS partition to my linux machine.. Hope it clarifies the doubt.. – Spleeny

Answer 1

Assuming you are using ntfs-3g to mount your NTFS partitions on linux (this is default on most current linux distributions).

You will need to read file extended attributes (see attr(5)), you can use pyxattr for this. NTFS attributes are stored in system.ntfs_attrib extended attribute as a set of flags which values are documented in ntfs-3g documentation.

Here is a sample code to read and decode NTFS file system attributes and use them to filter files:

import os, struct, xattr

# values from http://www.tuxera.com/community/ntfs-3g-advanced/extended-attributes/
attributes = ['readonly', 'hidden', 'system', 'unknown',
              'unknown', 'archive', 'unknown', 'unknown',
              'temp', 'unknown', 'unknown', 'compressed_dir',
              'offline', 'not_content_indexed' ] + ['unknown']*18

def ntfs_attr(path):
    attr = struct.unpack("i", xattr.get(path, "system.ntfs_attrib"))[0]
    for shift, attribute in enumerate(attributes):
        if (attr >> shift) & 1 == 1:
            yield attribute

def main():
    import sys
    if len(sys.argv) != 3:
        print "Usage: %s path attribute" % sys.argv[0]
        a = set(attributes)
        a.remove('unknown')
        print "where attribute is one of:", ' '.join(a)
        sys.exit(1)

    path = sys.argv[1]
    attribute = sys.argv[2]
    print "Files with %s attribute in %s:" % (attribute, path)
    for filename in os.listdir(path):
        fullname = os.path.join(path, filename)
        if attribute in ntfs_attr(fullname):
            print fullname


if __name__ == '__main__':
    main()

Phantasmal

Answer 2

There seems to be no python interface for NTFS attributes under linux.

NTFS-3G supports NTFS file attributes and exposes them for the linux tools getfattr and setfattr to read and set.

You can use python's subprocess to invoke getfattr and then parse the output.

Note: on my ubuntu system i had to install the package attr to get the commands getfattr and setfattr.

Obbard

just want to ask you before testing as to 'winsys' will work on linux machine? – Spleeny

@mahendraliya: The WinSys package wraps around the windows API, so it certainly can't be used under linux. I rewrote my answer. – Obbard

Why bother with external commands (slow for large number of calls!) if there is a python module for reading extended attributes? Your answer also contains a factual error: ntfs file attributes are supported by ntfs-3g almost from the very beginning (as system.ntfs_attrib attribute). What is available only in ntfs-3g-2011.1.15 or later is system.ntfs_attrib_be attribute, which is just an endianness-fixed version of system.ntfs_attrib. – Phantasmal

@abbot: I agree that is better to use the python module. I did not know that there is one. I do not agree that there is a factual error in my post. The attribute system.ntfs_attrib_be is for the commands getfattr and setfattr, and that is exactly what I wrote in my answer. – Obbard

You can also use system.ntfs_attrib with getfattr/setfattr without any problems. – Phantasmal

Answer 3

If your question is not limited to Python, you can try my example implemented in shell script.

This is also based on system.ntfs_attrib_be attribute in NTFS-3G. If you are just going to use it and don't care about how it is implemented (in Python or shell), just download it, install getfattr and setfattr from your distro, and use it.

https://gist.github.com/Explorer09/ac4bf6838c271a9968b3

Bucovina