How to ignore hidden files using os.listdir()?

Question

My python script executes an os.listdir(path) where the path is a queue containing archives that I need to treat one by one.

The problem is that I'm getting the list in an array and then I just do a simple array.pop(0). It was working fine until I put the project in subversion. Now I get the .svn folder in my array and of course it makes my application crash.

So here is my question: is there a function that ignores hidden files when executing an os.listdir() and if not what would be the best way?

Transsonic

you may also want to avoid folders with os.path.isdir() – Tirewoman

Answer 1

You can write one yourself:

import os

def listdir_nohidden(path):
    for f in os.listdir(path):
        if not f.startswith('.'):
            yield f

Or you can use a glob:

import glob
import os

def listdir_nohidden(path):
    return glob.glob(os.path.join(path, '*'))

Either of these will ignore all filenames beginning with '.'.

Partite

the proposed function listdir_nohidden is not quite compatible with os.listdir, since the use of yield makes it a generator. Instead it should run through the output list from os.listdir and remove entries that start with '.' – Galarza

@0sh: Why does it have to remove things in-place? Just define a new function that does list(listdir_nohidden(path)) and that new function is exactly compatible with os.listdir. – Froude

So in other words, there's no builtin regex in listdir() that allows you to customise your list. – Earthnut

Answer 2

This is an old question, but seems like it is missing the obvious answer of using list comprehension, so I'm adding it here for completeness:

[f for f in os.listdir(path) if not f.startswith('.')]

As a side note, the docs state listdir will return results in 'arbitrary order' but a common use case is to have them sorted alphabetically. If you want the directory contents alphabetically sorted without regards to capitalization, you can use:

sorted((f for f in os.listdir() if not f.startswith(".")), key=str.lower)

(Edited to use key=str.lower instead of a lambda)

Nildanile

key=lambda f: f.lower() can be written without lambda: key=str.lower – Anlace

To combine both: sorted([f for f in os.listdir('./') if not f.startswith('.')], key=str.lower) – Critter

Answer 3

On Windows, Linux and OS X:

if os.name == 'nt':
    import win32api, win32con


def folder_is_hidden(p):
    if os.name== 'nt':
        attribute = win32api.GetFileAttributes(p)
        return attribute & (win32con.FILE_ATTRIBUTE_HIDDEN | win32con.FILE_ATTRIBUTE_SYSTEM)
    else:
        return p.startswith('.') #linux-osx

Wesson

should also work on a Mac, hidden files start there with '.' as well. – Tactless

This is the only portable answer, great job, but the other answers provide a full wrapper for os.listdir so... [f for f in os.listdir(path) if not folder_is_hidden(f)] – Commode

Why do you check the FILE_ATTRIBUTE_SYSTEM flag too? Are system files hidden by default on Windows? – Loess

Answer 4

Joshmaker has the right solution to your question.
How to ignore hidden files using os.listdir()?

In Python 3 however, it is recommended to use pathlib instead of os.

from pathlib import Path 
visible_files = [
    file for file in Path(".").iterdir() if not file.name.startswith(".")
]

Cocksure

Answer 5

glob:

>>> import glob
>>> glob.glob('*')

(glob claims to use listdir and fnmatch under the hood, but it also checks for a leading '.', not by using fnmatch.)

Biradial

Answer 6

I think it is too much of work to go through all of the items in a loop. I would prefer something simpler like this:

lst = os.listdir(path)
if '.DS_Store' in lst:
    lst.remove('.DS_Store')

If the directory contains more than one hidden files, then this can help:

all_files = os.popen('ls -1').read()
lst = all_files.split('\n')

for platform independence as @Josh mentioned the glob works well:

import glob
glob.glob('*')

Langobardic

That works only if you have one hidden file, and you know the name of it. What if the directory contains dozens of hidden files, with arbitrary names that you can't know in advance? – Spalding

Hi @FeRD, Yes. When I am doing batch/backlog processing on a mac, I put all the files in new folder and .DS_Store gets created automatically. When I zip all the files and push it to a server .DS_Store also gets added. If there are various hidden files then may be you can try os.system('ls -1') – Langobardic

Not cross-platform. os.popen('ls -1').read() won't work on Windows. That's the whole point of os.listdir(). – Spalding

Answer 7

filenames = (f.name for f in os.scandir() if not f.name.startswith('.'))

Angellaangelle

Answer 8

You can just use a simple for loop that will exclude any file or directory that has "." in the front.

Code for professionals:

import os

directory_things = [i for i in os.listdir() if i[0] != "."] # Exclude all with . in the start

Code for noobs

items_in_directory = os.listdir()
final_items_in_directory = []

for i in items_in_directory:
    if i[0] != ".": # If the item doesn't have any '.' in the start
        final_items_in_directory.append(i)

Klockau