PowerShell history: how do you prevent duplicate commands?

Question

Background:

PowerShell history is a lot more useful to me now that I have a way to save history across sessions.

# Run this every time right before you exit PowerShell
get-history -Count $MaximumHistoryCount | export-clixml $IniFileCmdHistory;

Now, I am trying to prevent PowerShell from saving duplicate commands to my history.

I tried using Get-Unique, but that doesn't work since every command in the history is "unique", because each one has a different ID number.

Duroc

There's Set-PSReadlineOption -HistoryNoDuplicates, but it doesn't seem to work for me. – Magnus

Answer 1

Get-Unique also requires a sorted list and I assume you probably want to preserve execution order. Try this instead

Get-History -Count 32767 | Group CommandLine | Foreach {$_.Group[0]} |
Export-Clixml "$home\pshist.xml"

This approach uses the Group-Object cmdlet to create unique buckets of commands and then the Foreach-Object block just grabs the first item in each bucket.

BTW if you want all commands saved to a history file I would use the limit value - 32767 - unless that is what you set $MaximumHistoryCount to.

BTW if you want to automatically save this on exit you can do this on 2.0 like so

Register-EngineEvent PowerShell.Exiting {
  Get-History -Count 32767 | Group CommandLine |
  Foreach {$_.Group[0]} | Export-CliXml "$home\pshist.xml" } -SupportEvent

Then to restore upon load all you need is

Import-CliXml "$home\pshist.xml" | Add-History

Calk

hmm ... thanks for the post, but did you leave out the part where assign to $hist? – Duroc

Sorry, I was just picking a variable name other than $IniFileCmdHistory in order to avoid the dreaded horizontal scrollbar. :-) – Calk

What about just Get-History | Select -Unique? That works for me here at least and avoids the grouping and foreach (where I had to look twice before understanding what you do there :-)) – Wingfield

That's what I originally had put as my answer but the resulting object isn't a full fledged HistoryInfo object so when you try to rehydrate the history from file using "Import-Clixml <path> | Add-History" you get an error. – Calk

@Keith does this command prevent duplicate commands, or remove them? In other words, when I type ls 80 times, do I have to press up arrow 80 times to return to the command before "ls"? Or just twice? – Dexterous

It doesn't prevent duplicate commands in the current session but it will eliminate them from the previous session's commands that you load into a new session. – Calk

Thanks for the great tip, Keith. I would make one minor tweak, though: if (Test-path $MyHistoryFile) { Import-CliXml $MyHistoryFile | Add-History } That avoids the error message that will occur the first time one starts PowerShell after editing one's profile. – Cristie

didn't work for me this way. I found this link which did the trick for me: software.intel.com/en-us/blogs/2014/06/17/… – Golightly

Answer 2

The following command works for PowerShell in Windows 10 (tested in v.1803). The option is documented here.

Set-PSReadLineOption –HistoryNoDuplicates:$True

In practice, calling PowerShell with the following command (e.g. saved in a shortcut) opens PowerShell with a history without duplicates

%SystemRoot%\system32\WindowsPowerShell\v1.0\powershell.exe -NoExit -Command Set-PSReadLineOption –HistoryNoDuplicates:$True

Egyptian

Simply putting Set-PSReadLineOption -HistoryNoDuplicates in my $PROFILE works now. Not sure if it was a problem with earlier versions or not. – Lundin

Using a PowerShell profile also works. It requires creating it and changing the default execution policy which prevents running scripts. – Egyptian

-HistoryNoDuplicates works but if you have multiple flags (weirdly?) the order matters and only works when it comes last. for example this works Set-PSReadLineOption -PredictionSource HistoryAndPlugin -PredictionViewStyle ListView –HistoryNoDuplicates but this won't Set-PSReadLineOption –HistoryNoDuplicates -PredictionSource HistoryAndPlugin -PredictionViewStyle ListView – Mauro

Answer 3

Not directly related to duplicates, but similarly useful, this AddToHistoryHandler script block in my $PROFILE keeps short and simple commands out of my history:

$addToHistoryHandler = {
    Param([string]$line)
    if ($line.Length -le 3) {
        return $false
    }
    if (@("exit","dir","ls","pwd","cd ..").Contains($line.ToLowerInvariant())) {
        return $false
    }
    return $true
}
Set-PSReadlineOption -AddToHistoryHandler $addToHistoryHandler

Lundin

Helpful. Thanks. 👍 – Marindamarinduque