Getting compgen to include slashes on directories when looking for files

Question

I'd like to get the following behavior from my custom completion

Given

$ mkdir foo
$ touch foo faz/bar faz/baz

I'd like to get this

$ foo -u <tab><tab> =>
foo faz/

$ foo -u fa<tab><tab> =>
foo -u faz/

$ foo -u faz/<tab><tab> =>
bar baz

I assumed that compgen -f f would output foo faz/, but it outputs foo faz which doesn't help me much.

Do I need to post-process the output or is there some magic combination of options to compgen that work?

Ineptitude

Answer 1

I ran into the same problem. Here's the workaround I'm using:

1. Register the completion function with -o default, e.g., complete -o default -F _my_completion.
2. When you want to complete a filename, just set COMPREPLY=() and let Readline take over (that's what the -o default does).

There's a potential problem with this -- you might be using COMPREPLY=() to deny completion where it's not appropriate, and now that won't work anymore. In Bash 4.0 and above, you can work around this using compopt as follows:

1. At the top of your completion function, always run compopt +o default. This disables Readline filename completion when COMPREPLY is empty.
2. When you want to complete a filename, use compopt -o default; COMPREPLY=(). In other words, enable Readline filename completion only when you need it.

I haven't figured out a full workaround for pre-4.0 Bash, but I have something that works well enough for me. I can describe this if anyone really cares, but hopefully these older Bash versions will soon fall out of common use anyway.

Sacculate

Thanks!! This helped me. Interestingly the plus option (compopt +o) seems not to be documented in the bash manual.. – Bishopric

Glad it helped. BTW, here is a thread where the author of Bash tells me that my approach above is pretty much the way to do it (unfortunately): lists.gnu.org/archive/html/bug-bash/2013-10/msg00002.html – Sacculate

Answer 2

Previous answers require bash 4, or aren't composable with other compgen-based completions for the same command. The following solution doesn't require compopt (so it works with bash 3.2) and it is composable (for example you can easily add additional filtering to only match certain file extensions). Skip to the bottom if impatient.

---

You can test compgen by running it directly on the command line:

compgen -f -- $cur

where $cur is the word you would've typed so far during interactive tab-completion.

If I'm in a directory that has a file afile.py and a sub-directory adir, and cur=a then the above command shows the following:

$ cur=a
$ compgen -f -- $cur
adir
afile

Note that compgen -f shows files and directories. compgen -d shows only directories:

$ compgen -d -- $cur
adir

Adding -S / will add a trailing slash to every result:

$ compgen -d -S / -- $cur
adir/

Now, we can try listing all files and all directories:

$ compgen -f -- $cur; compgen -d -S / -- $cur
adir
afile.py
adir/

Note that there's nothing stopping you from calling compgen more than once! In your completion script you would use it like this:

cur=${COMP_WORDS[COMP_CWORD]}
COMPREPLY=( $(compgen -f -- "$cur"; compgen -d -S / -- "$cur") )

Unfortunately, this still doesn't give us the behaviour that we want, because if you type ad<TAB> you have two possible completions: adir and adir/. So ad<TAB> will complete to adir, at which point you still have to type in the / to disambiguate.

What we need now is a function that will return all files but no directories. Here it is:

$ grep -v -F -f <(compgen -d -P '^' -S '$' -- "$cur") \
>     <(compgen -f -P '^' -S '$' -- "$cur") |
>     sed -e 's/^\^//' -e 's/\$$//'
afile.py

Let's break it down:

• grep -f file1 file2 means show the lines in file2 that match any of the patterns in file1.
• -F means the patterns in file1 must match exactly (as a substring); they aren't regular expressions.
• -v means invert the match: Only show lines that aren't in file1.
• <(...) is bash process substitution. It allows you to run any command in the places where a file is expected.

So we're telling grep: here's a list of files -- remove any that match this list of directories.

$ grep -v -F -f <(compgen -d -P '^' -S '$' -- "$cur") \
>     <(compgen -f -P '^' -S '$' -- "$cur")
^afile.py$

I've added beginning and end markers with compgen's -P ^ and -S '$' because grep's -F does substring matching and we don't want to remove a filename like a-file-with-adir-in-the-middle just because its middle part matched a directory's name. Once we have the list of files we remove those markers with sed.

Now we can write a function that does what we want:

# Returns filenames and directories, appending a slash to directory names.
_mycmd_compgen_filenames() {
    local cur="$1"

    # Files, excluding directories:
    grep -v -F -f <(compgen -d -P ^ -S '$' -- "$cur") \
        <(compgen -f -P ^ -S '$' -- "$cur") |
        sed -e 's/^\^//' -e 's/\$$/ /'

    # Directories:
    compgen -d -S / -- "$cur"
}

You use it like this:

_mycmd_complete() {
    local cur=${COMP_WORDS[COMP_CWORD]}
    COMPREPLY=( $(_mycmd_compgen_filenames "$cur") )
}
complete -o nospace -F _mycmd_complete mycmd

Note that -o nospace means you don't get a space after a directory's /. For normal files we added a space at the end with sed.

One nice thing about having this in a separate function is that it's easy to test! For example here's an automated test:

diff -u <(printf "afile.py \nadir/\n") <(_mycmd_compgen_filenames "a") \
    || { echo "error: unexpected completions"; exit 1; }

Autogiro

I used it as: COMPREPLY=( $(_my_compgen_filenames "${cur}") ) but the space substitution in sed doesn't seem to happen. I think the sed works as expected, but the command substitution in my usage causes it to be gobbled (?). Any clues? – Alchemize

@Alchemize what are your symptoms -- when you press TAB it doesn't add a space to the end of a filename that is unique? What happens if you run _my_compgen_filenames "whatever-you-typed-before-pressing-tab" -- does it have a space at the end or not? Are you using complete with -o nospace? – Agripinaagrippa

Correct, it doesn't add a space at the end of a unique filename. If I run the function manually at the prompt, it it does add the space correctly.. I do have -o nospace, but I think I need that? How else do I stop it from adding a space for every completion? – Alchemize

@Alchemize yes you need -o nospace. What version of bash are you using? I tested this with bash 4.3.46. – Agripinaagrippa

I'm on 4.3.42 (from Fedora 24) – Alchemize

This answer is the most configurable answer I've found on the SE network. I was able to massage it into suggesting files and directories, including in subdirectories, of a specific directory. Thank you! – Ddene

Answer 3

This behaviour is influenced by the readline variable mark-directories (and the related mark-symlinked-directories). I believe the variable should be set on for complete to print a trailing slash on directory names (the default in bash v3.00.16). Seemingly the related behaviour of compgen doesn't append a slash to directory names :-\

Set the value of mark-directories alternately to on and off then retry your test:-

bind 'set mark-directories on'
bind 'set mark-directories off'

To make the change permanent for future invocations of bash, add the following to your INPUTRC file, commonly ~/.inputrc:-

$if Bash
# append '/' to dirnames
set mark-directories on
$endif

The tip to set readline variables in the current shell was found here: https://unix.stackexchange.com/a/27545. I didn't determine how to test the current value of a readline variable.

Additional Ideas

Perhaps only academic interest...

Create a list of directory names only and append a slash:-

compgen -d -S / f

Kennykeno

Hm, I think it's already set: bind -V|grep mark-directories => mark-directories is set to `on'. Tried switching it but no luck as far as I can tell. – Vocalize

@trygvis You are correct, I see your behaviour when using compgen. I initially performed my testing by defining a compspec with complete which did generate the trailing slash. Hmmm, don't have a further answer at this stage, though I've corrected the above text. – Kennykeno

I was getting a similar problem on the Mac, mark-directories was on, still got the problem. After I set mark-symlinked-directories it worked. It just happened that I was tabbing frequently into a dropbox-symlinked directory, and apparently it doesn't add the / even in deeper parts of the path that are not symlinks themselves. – Amble

Answer 4

If you want to be able to continue using tab after the directory was completed; here's the complete solution:

cur=${COMP_WORDS[COMP_CWORD]}
COMPREPLY=( $(compgen -S "/" -d -- ${cur}) ); 
compopt -o nospace;

first add a trailing slash to the answer. Then run the compopt -o nospace command to remove the trailing space. Now if you have only foo/bar in your directory, two tabs will suffice to type them!

Coagulate

Answer 5

Use the -o filenames option passed to the complete directive. As long as the contents of COMPREPLY are valid paths the slash will be appended as appropriate while still adding a space after completion of non-directory matches. (Works on bash 3.2.)

_cmd() {
  COMPREPLY=( $( compgen -f -- "$cur" ))
  COMPREPLY=( $( compgen -A file -- "$cur" ))  # same thing
}

complete -o filenames -F _cmd cmd

See info -n "(bash)Programmable Completion Builtins":

complete

... -o COMP-OPTION ...

     filenames
           Tell Readline that the compspec generates filenames, so
           it can perform any filename-specific processing (like
           adding a slash to directory names or suppressing
           trailing spaces).  This option is intended to be used
           with shell functions specified with `-F'.

And:

-A ACTION

... 'file'

           File names.  May also be specified as -f.

Note: compgen and complete have the same arguments.

Strade

Answer 6

This worked for me:

_my_completion() { 
  compopt -o nospace 
  COMPREPLY=( $(compgen -S "/" -d "${COMP_WORDS[$COMP_CWORD]}") )
}

complete -F _my_completion foo

Coffeepot