Tcsh and/or bash directory completion with variable hidden root prefix

Question

I'm trying to set up directory completion in tcsh and/or bash (both are used at my site) with a slight twist: for a particular command "foo", I'd like to have completion use a custom function to match the first /-delimited term to an actual subtree node, and then follow normal directory completion for any successive terms. It is sort of a combination of cdpath and completion, or I suppose a form of directory completion where the starting point is controlled by the completion script. It would work as follows:

$ foo xxx<TAB>
(custom completion function produces choices it finds at arbitrary levels in the dir tree)
xxxYYY xxxZZZ xxxBLAH ...
foo xxxYYY/<TAB>
(normal directory completion proceeds from this point on, to produce something like:)
foo scene/shot/element/workspace/user/...

We'd like to do this because we have a large production development tree (this is a CGI production facility), that shell-savvy users are navigating and jumping around in all the time. The complaint is that the upper levels of the tree are cumbersome and redundant; they just need a quick search on the first term to find possible "head" choices and do directory completion from there. It seems like programmable completion could offer a way to do this, but it is turning out to be pretty elusive.

I've made several attempts of custom bash and tcsh completion to do this, but the closest I've gotten is a form of "word completion" where the user must treat the directory levels as separate words with spaces (e.g. foo scene/ shot/ element/ workspace/ ...). I could continue hacking at my current scripts--but I've been wondering if there's something I'm not understanding--this is my first attempt to program completion, and the docs and examples are pretty thin in shell books and on the internet. If there's any completion-guru's out there that can get me on the right track, I'd appreciate it.

FWIW: here is what I've got so far (in tcsh first, then bash). Note that the static root '/root/sub1/sub2/sub3' is just a placeholder for a search function that would find different matches in different levels. If I can get that to work, I can sub in the search feature later. Again, both examples do word completion, which requires user to type a space after each matching term (I also have to remove the spaces in the function to construct an actual path, yuck!)

TCSH EXAMPLE (note the function is actually a bash script):

complete complete_p2 'C@*@`./complete.p2.list.bash $:1 $:2 $:3 $:4 $:5 $:6 $:7 $:8 $:9`@@'

#!/bin/bash --norc

# complete.p2.list.bash - Completion prototype "p2" for shotc command

# Remove spaces from input arguments
ppath=`echo $@ | sed -e 's/ //g'`

# Print basenames (with trailing slashes) of matching dirs for completion
ls -1 -d /root/sub1/sub2/sub3/$ppath* 2>/dev/null | sed -e 's#^.*/##' | awk '{print $1 "/"}'

BASH EXAMPLE:

_foo() 
{
    local cur prev opts flist
    COMPREPLY=()
    cur="${COMP_WORDS[COMP_CWORD]}"
    prev="${COMP_WORDS[COMP_CWORD-1]}"

    # Get all command words so far (omit command [0] element itself), remove spaces
    terms=`echo ${COMP_WORDS[@]:1} | sed -e 's/ //g'`

    # Get basenames (with trailing slashes) of matching dirs for completion
    flist=`ls -1 -d /root/sub1/sub2/sub3/${terms}* 2>/dev/null | sed -e 's#^.*/##' | awk '{print $1 "/"}' | xargs echo`

    COMPREPLY=( $(compgen -W "${flist}" ${cur}) )
    return 0
}
complete -F _foo foo

Dimitry

Would bash's CDPATH environment variable work for you? – Aircondition

I looked into CDPATH, but it doesn't work WITH completion. You can "cd name", but not "cd name<TAB> ..." I really need the completion. – Dimitry

I thought bash completion (the extra script full of useful goodies that ships with bash on most distros) included an updated definition of cd completion using CDPATH. – Aircondition

@Jefromi: You're right and it seems to be capable of doing what I think Jeremy wants. – Roussillon

Thanks for the tip, I'll check it out. It is worth mentioning however, that I need to do more than cd; setting up subtree-specific environments on the fly for instance. I was planning on using a custom command, but maybe I can just alias cd to something else (that may be too intrusive though). – Dimitry

Answer 1

This seems like it might do what you're looking for:

_foo()
{
    local cur prev opts flist lastword new
    COMPREPLY=()
    cur="${COMP_WORDS[COMP_CWORD]}"
    prev="${COMP_WORDS[COMP_CWORD-1]}"
    lastword="${COMP_WORDS[@]: -1}"

    if [[ $lastword =~ / ]]
    then
        new="${lastword##*/}"      # get the part after the slash
        lastword="${lastword%/*}"  # and the part before it
    else
        new="${lastword}"
        lastword=""
    fi

    flist=$( command find /root/sub1/sub2/sub3/$lastword \
      -maxdepth 1 -mindepth 1 -type d -name "${new}*" \
      -printf "%f\n" 2>/dev/null )

    # if we've built up a path, prefix it to 
    #   the proposed completions: ${var:+val}
    COMPREPLY=( $(compgen ${lastword:+-P"${lastword}/"} \
      -S/ -W "${flist}" -- ${cur##*/}) )
    return 0
}
complete -F _foo -o nospace foo

Notes:

• I think one of the keys is the nospace option
• I feel like I've reinvented a wheel somewhere in the function above, possibly by not using $COMP_POINT
• You're not (yet, at least) using $prev (which always maintains the value "foo" in my function)
• Readability and functionality can be improved by using $() instead of backticks
• You should use command to prevent running aliases and such: flist=$(command ls -1 -d...
• I'm using find instead of ls because it's better suited
• You can add the slash using -S/ with compgen instead of your awk command
• You can use $cur instead of $terms since you don't have to strip out spaces, but I'm using $lastword and $new (two new variables)
• It's not necessary to use xargs echo since an array with newlines works fine
• I have not tested this with directory names having spaces or newlines in them

Roussillon

Thanks Dennis! That is very close to what I'm looking for. I may be able to use it with a few tweaks, or maybe even as is. I'll see if I can apply a similar method to tcsh as well. – Dimitry

I thought I was pretty alone to need complex completion scripts :) +1 for the code effort ! – Dachy

Answer 2

my solution, which is admittedly an 800-lb hammer, was to write a perl script to handle the completion the way i wanted it to. in tcsh...

complete cd 'p|1|`complete_dirs.pl $:1 $cdpath`|/'

#!/usr/bin/perl

my $pattern = shift @ARGV;
my @path = @ARGV;
my @list;

if ($pattern =~ m!^(/.+/|/)(.*)!) {
  @list = &search_dir($1,$2,undef);
} elsif ($pattern =~ m!(.+/|)(.*)!) {
  my $dir; foreach $dir ('.',@path) {
    push(@list,&search_dir("$dir/$1",$2,$1));
  }
}
if (@list) {
  @list = map { &quote_path($_) } @list;
  print join(' ',@list), "\n";
}

sub search_dir {
  my ($dir,$pattern,$prefix) = @_;
  my @list;

  if (opendir(D,$dir)) {
    my $node; while ($node = readdir(D)) {
      next     if ($node =~ /^\./);
      next unless ($node =~ /^$pattern/);
      next unless (-d "$dir$node");

      my $actual; if (defined $prefix) {
        $actual = "$prefix$node";
      } elsif ($dir =~ m!/$!) {
        $actual = "$dir$node";
      } else {
        $actual = "$dir/$node";
      }
      push(@list,$actual);
    }
    closedir(D);
  }
  return @list;
}
sub quote_path {
  my ($string) = @_;

  $string =~ s!(\s)!\\$1!g;
  return $string;
}

Opheliaophelie

Answer 3

So, here is a tcsh solution. Adds full $cdpath searching for autocompleting directories. The complete command is:

complete cd 'C@/@d@''p@1@`source $HOME/bin/mycdpathcomplete.csh $:1`@/@'

I am a bit of a tcsh hack, so the string manipulation is a bit crude. But it works with negligible overhead... mycdpathcomplete.csh looks like this:

#!/bin/tcsh -f

set psuf=""
set tail=""

if ( $1 !~ "*/*" ) then
    set tail="/$1"
else
    set argsplit=(`echo "$1" | sed -r "s@(.*)(/[^/]*\')@\1 \2@"`)
    set psuf=$argsplit[1]
    set tail=$argsplit[2]
endif

set mycdpath=(. $cdpath)
set mod_cdpath=($mycdpath)

if ($psuf !~ "") then
    foreach i (`seq 1 1 $#mycdpath`)
        set mod_cdpath[$i]="$mycdpath[$i]/$psuf"
        if ( ! -d $mod_cdpath[$i] ) then
            set mod_cdpath[$i]=""
        endif
    end
endif

# debug statements
#echo "mycdpath = $mycdpath"
#echo "mod_cdpath = $mod_cdpath"
#echo "tail = $tail"
#echo "psuf = $psuf"

set matches=(`find -L $mod_cdpath -maxdepth 1 -type d -regex ".*${tail}[^/]*" | sed -r "s@.*(/?${psuf}${tail}[^/]*)\'@\1@" | sort -u`)

# prune self matches
if ($psuf =~ "") then
    foreach match (`seq 1 1 $#matches`)
        set found=0
        foreach cdp ($mod_cdpath)
            if ( -e "${cdp}${matches[$match]}" ) then
                set found=1;
                break;
            endif
        end
        if ( $found == 0 ) then
            set matches[$match]=""
        else
            set matches[$match]=`echo "$matches[$match]" | sed -r "s@^/@@"`
        endif
    end
endif

echo "$matches"

Skywriting

Answer 4

You could just make a symlink to the first interesting node in the tree. I've done this in the past when I couldn't be bothered auto-completing large directory trees.

Naturalistic

Unfortunately the "first interesting node" can vary greatly. As the users spend time in different parts of the tree and hop around the subtrees, there is no predicting which levels they may anchor a completion attempt. Unless we link every level (seems cumbersome and unwieldy), I'm not sure links will solve it. – Dimitry

Answer 5

The basic tcsh solution is very easy to implement as follows:

    complete foo 'C@*@D:/root/sub1/sub2/sub3@'

There is no bash script dependency required. Of course the base directory is hardwired in this example.

Mum