how can I avoid storing a command in ipython history?

Question

In bash I can prevent a command from being saved to bash history by putting a space in front of it. I cannot use this method in ipython. How would I do the equivalent in ipython?

Disadvantageous

Related: #19606775 – Raver

Answer 1

Obsolescence note: this was written for IPython 4.0.1. As of v5, IPython no longer uses readline.

---

No stock way, can be added in or worked around.

There are two kinds of history in IPython:

• readline history. It is available with special keys/key combinations (up/down, Ctrl-R etc). Only stores lines entered (with <Enter>) on the console (or, in pyreadline, pasted from the clipboard, too). IPython relies on a local implementation of the Python readline API which doesn't have a "do not add" function and normally adds every line to history.
  IPython has a facility to alleviate this, IPython.core.interactiveshell.ReadlineNoRecord, a context manager that makes a snapshot of history then restores it. As of ipython 4.0.1, it's only used by %run magic to avoid adding script interactive input to history.

• IPython history. It's saved in the DB and automatic variables. It contains complete inputs (readline grabs each <Enter>'ed line separately for multiline ones) and outputs. Saving is implemented in HistoryManager.store_inputs() (for inputs) and HistoryManager.store_output() (for outputs) which is called from InteractiveShell.run_cell and is governed by its store_history argument. The latter is in turn called from TerminalInteractiveShell.interact with store_history=True.

There are two ways to solve the problem for either layer:

• prevent adding the input in the first place. This cannot be done with a magic prefixed to a command, only with one that is run as a separate command and toggles a flag. That's because, as you've seen, the current input has already been stored by the time a magic command gets control.

  • readline: there's no relevant entry in the public API, so it's implementation-specific. E.g. for pyreadline, adding is done with pyreadline.modes.basemode.BaseMode.add_history(). The mode object is accessible as get_ipython().readline.rl.mode.
  • Decorating run_cell and add_history with wrappers checking flags in corresponding objects and a custom magic command that sets/toggles them should do the trick.

• automatically remove evidence input/output from history immediately after execution. This can be done with a prefix magic.

  • IPython: HistoryManager doesn't have any facilities to remove entries (neither from DB nor from variables). Alas, hacking the DB by hand/replacing stock HistoryManager is necessary. Also note that the class has an optional cache of HistoryManager.db_cache_size (disabled by default).
  • readline: remove_history_item(index) is in the API. You need to know the number of lines in the input.

Alternatively, if you only require this to enter passwords, consider other ways that don't echo the password on screen (thus not making it a part of console history):

• getpass.getpass()
• storing it elsewhere (like a configuration file only readable by you)

Jat

1. "prevent adding the input in the first place. This cannot be done with a magic prefixed to a command, only with one that is run as a separate command and toggles a flag" what is an example of a separate command toggling a flag to prevent adding the input? 2. Decorating run_cell and add_history with wrappers checking flags in corresponding objects and a custom magic command that sets/toggles them should do the trick. Can you add an example? – Raver

@Raver 0) see obsolescence note; 1)%toggle,your_command,%toggle; 2) this means to replace those functions with wrappers around them; an example is beyond the scope of the answer, this should be clear to anyone with decent expertize. – Jat

Answer 2

I think I finally figure this out. This method not really 'prevent' the ipython to record history but actually delete the history record. But I think still it is a good way to achieve this goal.

ipython history is stored in a sqlite database file in $(ipython locate)/profile_default/history.sqlite.

The database table History has four columns: session, line, source and source_raw. The session column presents for the session_id of current session, it can be get with the method in ipython: get_ipython().history_manager.hisdb.get_last_session_id() in ipython. The line column is just the line num.

So what I'm going to do is I'm going to delete this record in the database within ipython enviroment:

1) The history db object can be get using get_ipython().history_manager.db in ipython

hismgr = get_ipython().history_manager   

2) get the session id:

session_id = hismgr.get_last_session_id()

3) delete the history line with the line_id (assume it is 111 here) and session_id

hismgr.db.execute('DELETE FROM history WHERE line={0} and session={1}'.format(111, session_id))

4) the In and Out Array List is also kind like history, but it stored in memory, so it could be sweep or rewrite like variable.

In[111]=Out[111]=''

To 'prevent a command from being saved to ipython history', means delete the line's history record in database and and rewrite In and Out Array in ipython enviroment. And we could combine these four lines into to one to do the job one time for all. Here is an example if we want to delete line 128:

In [127]: history -l 3 -n
 124: history -l 3 -n
 125: hismgr = get_ipython().history_manager; session_id = hismgr.get_last_session_id();hismgr.db.execute('DELETE FROM history WHERE line={0} and session={1}'.format(123, session_id))
 126: history -l 3 -n

In [128]: passwd='123456'

In [129]: hismgr = get_ipython().history_manager; session_id = hismgr.get_last_session_id();hismgr.db.execute('DELETE FROM history WHERE line={0} and session={1}'.format(128, session_id));In[128]=Out[128]='';
Out[129]: <sqlite3.Cursor at 0x7f8dce3dae30>

In [130]: history -l 3 -n
 126: history -l 3 -n
 127: history -l 3 -n
 129: hismgr = get_ipython().history_manager; session_id = hismgr.get_last_session_id();hismgr.db.execute('DELETE FROM history WHERE line={0} and session={1}'.format(128, session_id));In[128]=Out[128]='';

The history line 128 is gone.

Here is some documents I went through

IPython.core.history.HistoryManager

python-sqlite3

Resolvable

This doesn't really "avoid storing a line in history", it rather deletes it from history post factum. – Jat

@Jat I know. But I read through the ipython document, and can't find some way really "prevent a command saved into history", So I come out that if I delete the line would do the same job. – Resolvable

Thanks. This was so extremely obtuse to find any information about. For anyone else just trying to delete their command history, create hismgr as written, but execute hismgr.db.execute('DELETE from history'). This will delete all of your history. – Eerie

Answer 3

I was looking for a solution myself and then realized that I could just input the secret. E.g., I used the below to initialize github API when using pygithub:

In [1]: gh = github.Github(base_url="https://api.github.com", login_or_token=input("token: "))
token: abcd

In [2]: %hist
gh = github.Github(base_url="https://api.github.com", login_or_token=input("token: "))
%hist

Grind

That looks perfectly valid, but keep in mind IPython also stores command output. Looking into my local SQLite db, the output table is empty though… but it seems ordinarily, it would contain the token you printed. – Peraza