How to write code to autocomplete words and sentences?

Question

I'd like to write code that does autocompletion in the Linux terminal. The code should work as follows.

It has a list of strings (e.g. "hello, "hi", "how are you", "goodbye", "great", ...).

In the terminal, the user will start typing and when there is some match possibility, he gets the hint for possible strings, from which he can choose (similarly as in vim editor or google incremental search).

e.g. he starts typing "h", and he gets the hint

h"ello"

_ "i"

_"ow are you"

And better yet would be if it would complete words not only from the beginning, but from an arbitrary part of the string.

Ichthyolite

What's the context? Are you running Python application in the terminal which receives input from the user? I don't get it... – Disjoin

@FelixKling Yes, exactly – Ichthyolite

Answer 1

(I'm aware this isn't exactly what you're asking for, but) If you're happy with the auto-completion/suggestions appearing on TAB (as used in many shells), then you can quickly get up and running using the readline module.

Here's a quick example based on Doug Hellmann's PyMOTW writeup on readline.

import readline

class MyCompleter(object):  # Custom completer

    def __init__(self, options):
        self.options = sorted(options)

    def complete(self, text, state):
        if state == 0:  # on first trigger, build possible matches
            if text:  # cache matches (entries that start with entered text)
                self.matches = [s for s in self.options 
                                    if s and s.startswith(text)]
            else:  # no text entered, all matches possible
                self.matches = self.options[:]

        # return match indexed by state
        try: 
            return self.matches[state]
        except IndexError:
            return None

completer = MyCompleter(["hello", "hi", "how are you", "goodbye", "great"])
readline.set_completer(completer.complete)
readline.parse_and_bind('tab: complete')

input = raw_input("Input: ")
print "You entered", input

This results in the following behaviour (<TAB> representing a the tab key being pressed):

Input: <TAB><TAB>
goodbye      great        hello        hi           how are you

Input: h<TAB><TAB>
hello        hi           how are you

Input: ho<TAB>ow are you

In the last line (HOTAB entered), there is only one possible match and the whole sentence "how are you" is auto completed.

Check out the linked articles for more information on readline.

---

"And better yet would be if it would complete words not only from the beginning ... completion from arbitrary part of the string."

This can be achieved by simply modifying the match criteria in the completer function, ie. from:

self.matches = [s for s in self.options 
                   if s and s.startswith(text)]

to something like:

self.matches = [s for s in self.options 
                   if text in s]

This will give you the following behaviour:

Input: <TAB><TAB>
goodbye      great        hello        hi           how are you

Input: o<TAB><TAB>
goodbye      hello        how are you

---

Updates: using the history buffer (as mentioned in comments)

A simple way to create a pseudo-menu for scrolling/searching is to load the keywords into the history buffer. You will then be able to scroll through the entries using the up/down arrow keys as well as use Ctrl+R to perform a reverse-search.

To try this out, make the following changes:

keywords = ["hello", "hi", "how are you", "goodbye", "great"]
completer = MyCompleter(keywords)
readline.set_completer(completer.complete)
readline.parse_and_bind('tab: complete')
for kw in keywords:
    readline.add_history(kw)

input = raw_input("Input: ")
print "You entered", input

When you run the script, try typing Ctrl+r followed by a. That will return the first match that contains "a". Enter Ctrl+r again for the next match. To select an entry, press ENTER.

Also try using the UP/DOWN keys to scroll through the keywords.

Chalybeate

Thank you, I tested the code. It only has missing some features I'd like to have. automatic autocomplation while typing, without the need to press tab key possibility to complete from arbitrary part of the strings, possibility of choosing between matches with rows, or other keyboard shortcuts. – Ichthyolite

I forgot that this feature is in google, so it's possible to try search something with google. – Ichthyolite

AFAIK there isn't a canned solution to do what you want. That level of interactivity in a console app would require a lot more work. As @jro commented in the other answer, curses would be a good start. I'm merely pointing out the more common approach to cmd-line-based autocompletion, i.e. using readline. Apologies for not being of more help. Good luck with your app. – Chalybeate

"I forgot that this feature is in google"... huh? What king of application are you trying to write? – Chalybeate

Just ordinary app to save me some typing. I hoped there are some libraries that could do this easily. Your answer is acceptable, but I will wait for some time yet if it can be improved yet. Thank you – Ichthyolite

Command line apps work quite different from GUIs/webpages. For menu-based interfaces as the ones in your linked screenshots, you'll have to write your app as a TUI using libraries such as ncurses (used by vim). The curses module provides an interface to the curses library, but you'll quickly find that its a completely different can of worms to your standard cmd-line app. – Chalybeate

But is ncurses neccessary for completion without the tab key? ncurses is needed I think for the feature with browsing through the list of suggestions, this is the feature I give up or leave for later. – Ichthyolite

That would not require ncurses, but I doubt if completion every keypress is doable using readline (quite possible for good reasons). The closest solution I can think of to "browsing through the list of suggestions" is to load the keywords into the readline history buffer which allows you to navigate through it using your up/down arrow keys as well as using the Ctrl-r search shortcuts. I'll update the answer with an example. – Chalybeate

Answer 2

To enable autocomplete in a Python shell, type this:

import rlcompleter, readline
readline.parse_and_bind('tab:complete')

(thanks to http://blog.e-shell.org/221)

Besot

Answer 3

You may want to checkout fast-autocomplete: https://github.com/seperman/fast-autocomplete

It has a demo mode that you can type and get results as you type: https://zepworks.com/posts/you-autocomplete-me/#part-6-demo

It is very easy to use:

>>> from fast_autocomplete import AutoComplete
>>> words = {'book': {}, 'burrito': {}, 'pizza': {}, 'pasta':{}}
>>> autocomplete = AutoComplete(words=words)
>>> autocomplete.search(word='b', max_cost=3, size=3)
[['book'], ['burrito']]
>>> autocomplete.search(word='bu', max_cost=3, size=3)
[['burrito']]
>>> autocomplete.search(word='barrito', max_cost=3, size=3)  # mis-spelling
[['burrito']]

Disclaimer: I wrote fast-autocomplete.

Skeg

This is actually very handy. Thank you sir! – Whinny

Sure! Happy to hear you found it handy! – Skeg

Just one doubt, is it scalable. I mean if my dictionary is large enough containing like some million or billion words, would it be able to return the results this fast ? – Knighthood

@Knighthood I have not had a chance to optimize it. If somebody has time to optimize it via Cython, that would be great. My use case at the time of creation did not require huge dictionaries. You can read more here at the bottom: github.com/seperman/fast-autocomplete/issues/10 – Skeg

I'm afraid this answer is not useful. The question asks about an auto-completer for the shell, not about an auto-completer in general. – Captain

@Captain If you look at the docs, it does offer shell autocomplete: github.com/seperman/fast-autocomplete#demo – Skeg

Oh, right, I overlooked that. Thanks! – Captain

Answer 4

I guess you will need to get a key pressed by the user.

You can achieve it (without pressing enter) with a method like this:

import termios, os, sys

def getkey():
    fd = sys.stdin.fileno()
    old = termios.tcgetattr(fd)
    new = termios.tcgetattr(fd)
    new[3] = new[3] & ~termios.ICANON & ~termios.ECHO
    new[6][termios.VMIN] = 1
    new[6][termios.VTIME] = 0
    termios.tcsetattr(fd, termios.TCSANOW, new)
    c = None
    try:
        c = os.read(fd, 1)
    finally:
        termios.tcsetattr(fd, termios.TCSAFLUSH, old)
    return c

Then, if this key is a tab key (for example, that's something you need to implement), then display all possibilities to user. If that's any other key, print it on stdout.

Oh, of course you will need to have getkey() looped in a while, as long as the user hits enter. You can also get a method like raw_input, that will get the whole word sign by sign, or display all the possibilities, when you hit a tab.

At least that's the item, you can start with. If you achieve any other problems, than write about them.

EDIT 1:

The get_word method can look like this:

def get_word():
    s = ""
    while True:
        a = getkey()
        if a == "\n":
            break
        elif a == "\t":
            print "all possibilities"
        else:
            s += a

    return s

word = get_word()
print word

The issue I'm occuring right now is the way to display a sign, you have just entered without any enteres and spaces, what both print a and print a, does.

Deferral

To display the suggestions, you could then use the curses module to be able to mimic vi's way of displaying the suggestions. – Koniology

Oh, didn't hear about this one before. Looks interesting, thanks :) – Deferral

@Deferral thank you. There is only the core part missing incremental matching and showing hints while typing – Ichthyolite

Answer 5

Steps:

1. Create a file .pythonrc in home directory by this command: vi .pythonrc

2. Enter this content:

   import rlcompleter, readline  
   readline.parse_and_bind('tab:complete') 
   

3. Close the file

4. Now run

   echo "export PYTHONSTARTUP=~/.pythonrc" >> ~/.bashrc

5. Restart the terminal

Deadly

Answer 6

For those (like me) that end up here searching for autocomplete in the interpreter:

https://web.archive.org/web/20140214003802/http://conjurecode.com/enable-auto-complete-in-python-interpreter/

This involves creating a file .pythonrc, modifying .bashrc and an import sys you have to import every time you launch the Python interpreter.

I wonder if the latter can be automated for even more win.

Cocainize