Recursively find files that are not publicly readable

Question

I'd like to recursively find all files in my public_html folder that are not publicly readable (i.e. those files that will cause 403 error). Is there a quick bash command for that? I am using Linux servers running apache, if that is relevant. Thanks.

Moderato

Try cURL – Valvulitis

Thanks @PatrickRoberts. Could you elaborate your answer please? By the way, I'm looking to do that on the server side (as a administrator to check which files might cause trouble) rather than on the client side (as a user by actually downloading all the files). – Moderato

Depends on what kind of server you're using then. You should specify that in your question. – Valvulitis

Answer 1

Use the find command:

find . ! -perm -o=r

Will search for files within the current directory and subdirectories that has a file permission so that the "others" group cannot read that file.

The manual page for find gives some examples of these options.

You can run this command as the www-data user:

find . ! -readable

To find all files that are NOT readable by the web server.

Imperial

Thanks a lot for your response, @cleod_ideafix. But find . -perm -o-r still returns me files that others group has readable access (it seems to return me all files in the folder). But find . -perm 700 and find . -perm 600 seem to work. – Moderato

@YingXiong: it should be find . ! -perm -o=r – Kidney

It's also worth pointing out that find . ! -perm -o=r is POSIX-compliant, whereas the -readable primary (test) is a GNU find extension. – Kidney

Great pointers, but please fix the fundamentally broken first command. – Kidney

Answer 2

^{Note: This answer was originally written while mcleod_ideafix's answer still contained the following broken command: find . -perm -o-r; the last section of this answer explains why it cannot work.}

find . ! -perm -o=r

• matches all files and directories in the current directory's subtree (.)
  • to restrict matching to files only, append -type f.
• which do not (!) have the read permission (r) set for security principal "others (world)" (o)

This works as intended, as long as all files examined were neither created by the user account in whose context the webserver runs nor belong to a group that the web-server account is a member of. Typically, that is the case.

The above command is POSIX-compliant.

---

mcleod_ideafix's answer offers a more robust option available with GNU find's (nonstandard) -readable test:

When run in the context of the webserver's user account (on Linux, www-data), this will only match files and directories that the webserver effectively cannot read, irrespective of what user and group own the file:

sudo -u www-data find . ! -readable -prune

Note that the -prune prevents attempts to descend into unreadable subdirectories and thereby suppresses warnings.

• If you want to restrict matching to files only, it gets more complicated:
  sudo -u www-data find . ! -readable \( -type f -print -o -prune \)

---

As for what doesn't work:

• Commands such as s find . -perm 700 and find . -perm 600 will only match files with that exact mode (700 translates to u=rwx,go=, 600 to u=rw,go=), so you'd have to construct commands for all possible variations for the user and group permissions to find all matches of interest.
• find . -perm -o-r is fundamentally broken and invariably matches any file or directory:
  • The - prefix of the value passed to -perm specifies that all the permissions that follow be set in matching files.
  • -perm only allows positive matching of permissions (what IS set as opposed to what's NOT set), so it is fundamentally impossible to express "match only if this permission is NOT set" logic with just a -perm argument only.
    • While -r is syntactically supported (because it is valid chmod syntax), it is pointless here and results in a no-op.
    • Technically, -o-r tells -perm to subtract (remove) the read-permission bit for "others" from the starting value of the mode mask used for matching; since that starting value is 000, or, symbolically, a=, any attempt to subtract permissions from that will be a no-op, i.e., have no effect. To put it in the immortal words of Billy Preston and Bruce Fisher: Nothin' from nothin' leaves nothin'
    • The net effect is that no restriction whatsoever is placed on the permissions of potentially matching files or directories, so that all items are unconditionally matched.
  • Thus, the only option is to have -perm itself match positively (-perm -o=r), and then negate the result by placing find's negation operator, !, before it.

Kidney