How to sort the output of winget list by column in powershell?
Asked Answered
D

8

5

I'm not getting the expected output when trying to sort the output from winget list in powershell. The Id column is not sorted.


# winget list | Sort-Object -Property Id

ScreenToGif                            NickeManarin.ScreenToGif               2.37.1                             winget
Microsoft Visual C++ 2015-2019 Redist… Microsoft.VCRedist.2015+.x64           14.28.29325.2           14.34.318… winget
paint.net                              {28718A56-50EF-4867-B4C8-0860228B5EC9} 4.3.8
Python 3.10.0 (64-bit)                 {21b42743-c8f9-49d7-b8b6-b5855317c7ed} 3.10.150.0
Microsoft Support and Recovery Assist… 0527a644a4ddd31d                       17.0.7018.4
-----------------------------------------------------------------------------------------------------------------------
Name                                   Id                                     Version                 Available  Source
Paint 3D                               Microsoft.MSPaint_8wekyb3d8bbwe        6.2009.30067.0
Microsoft .NET SDK 6.0.402 (x64)       Microsoft.DotNet.SDK.6                 6.0.402                            winget
3D Viewer                              Microsoft.Microsoft3DViewer_8wekyb3d8… 7.2010.15012.0
Microsoft Sticky Notes                 Microsoft.MicrosoftStickyNotes_8wekyb… 3.8.8.0

Q: How can I sort the output of winget list by the Id column in powershell?

I would like to see a powershell solution similar to the Bash sort -k <column-number>, to sort on any column. I fail to see why this obvious function is not available in powershell?

Drawee answered 1/11, 2022 at 13:5 Comment(6)
You are trying to sort a string-object. String doesn't have properties assigned to "columns". You will have to convert the string to an object with the required priorities first.Sinful
That obvious function is indeed Sort-Object. What's not being understood is that winget list does not return objects, it returns text. You can reorganize the output to return objects.Ellen
This seems like a promising script to parse the output of winget... ConvertFrom-WingetStdout.ps1Sinful
How can you check when some output is not returning objects?Drawee
@Drawee If it's not a powershell command, then it isn't outputting objects. Get-command would say the commandtype is application. .gettype() on the output would say string.Warm
A powershell command would follow the commonverb-noun convention.Warm
W
7

It outputs text, not an object with properties like "Id". This program's output isn't very smart. It looks like it outputs some special characters as well like (U+2026 HORIZONTAL ELLIPSIS). The first thing that occurs to me is to cut off the first 39 characters and then sort it by column 40 onward, where Id starts. That should be like sort -k in unix. I believe a powershell version of winget is coming in the future. Replacing non-ascii with spaces and skipping the first 4 lines.

# or -creplace '\P{IsBasicLatin}'
(winget list) -replace '[^ -~]',' ' | select-object -skip 4 | 
  sort-object { $_.substring(39) }

Python 3.10.0 (64-bit)                 {21b42743-c8f9-49d7-b8b6-b5855317c7ed} 3.10.150.0
paint.net                              {28718A56-50EF-4867-B4C8-0860228B5EC9} 4.3.8
Microsoft Support and Recovery Assist  0527a644a4ddd31d                       17.0.7018.4
Name                                   Id                                     Version                 Available  Source
ScreenToGif                            NickeManarin.ScreenToGif               2.37.1                             winget
Microsoft .NET SDK 6.0.402 (x64)       Microsoft.DotNet.SDK.6                 6.0.402                            winget
3D Viewer                              Microsoft.Microsoft3DViewer_8wekyb3d8  7.2010.15012.0
Microsoft Sticky Notes                 Microsoft.MicrosoftStickyNotes_8wekyb  3.8.8.0
Paint 3D                               Microsoft.MSPaint_8wekyb3d8bbwe        6.2009.30067.0
Microsoft Visual C++ 2015-2019 Redist  Microsoft.VCRedist.2015+.x64           14.28.29325.2           14.34.318  winget

Trying out the Cobalt module that uses Crescendo to parse Winget. There's no name property, and version is just a string (apparently these things are more of a challenge). There's a lot of guid's at the top.

install-module cobalt -scope currentuser
get-wingetpackage | sort id

ID                                      Version             Available Source
--                                      -------             --------- ------
{04F3299A-F322-45A6-8281-046777B9C736}  21.0.3
{0E8670B8-3965-4930-ADA6-570348B67153}  11.0.2100.60
{0EDB70B6-EEA7-413B-BBC4-89E2CD36EFDE}  11.5.18
#...
7zip.7zip                               21.07               22.01     winget
Acrylic Suite
Acrylic Wi-Fi Home
Warm answered 1/11, 2022 at 14:55 Comment(13)
First column isn't necessarily 39 characters in width. It depends on the size of the console window. A better approach would be to determine column width from the header line.Unsling
@Unsling That isn't my experience. Plus in the beginning there's 2 blank lines with 3 or 4 backspaces in them. I know sometimes the number of columns vary.Warm
I can reproduce this on Win 10 with both cmd and powershell consoles. Example row from PowerShell console: Visual Studio Community 2022 Microsoft.VisualStudio.2022.Community 17.2.6 17.3.6 winget -> 1st column of 89 charactersUnsling
Yes, I don't know why it insist setting the width of the 1st column to 40 chars (UTF-8 ... ellipsis included). Madness. @zett42, what powershell version are you using and what is the charset (codepage) and OutputEncoding for that?Drawee
Parsing the output of this program is a unique challenge indeed.Warm
I think when you pipe it to something it stays 40 characters, although there's some encoding issues.Warm
Interesting, winget uses a fixed column width only when redirected. So substring(39) actually seems to work. I'd only suggest to filter the header lines from the sorted output now. winget | select -skip 4 works for me.Unsling
@Unsling ok, thank you, I'm also replacing non-ascii with spacesWarm
Interestingly, I just noticed I got an exception with your 1st command: Sort-Object: Exception calling "Substring" with "1" argument(s): "startIndex cannot be larger than length of string. (Parameter 'startIndex')", even though it worked. And it is also really weird that you have to skip 4 when there are only 2 lines shown. Does windows EOL make \n and \r both count?Drawee
@Drawee winget outputs 2 blank lines first with backspaces and other characters.Warm
Weird, now it works for me only with substring(38).Unsling
@Unsling It should work with a column of spaces in the front.Warm
No, your command trims the 1st character from the ID column for me. Possibly depending on localization of the OS?Unsling
M
8

To complement the existing, helpful answers:

I would like to see a powershell solution similar to the Bash sort -k <column-number>, to sort on any column.
I fail to see why this obvious function is not available in powershell?

  • The sort utility does not sort by columns with -k (--key); it sorts by fields, with any non-empty run of whitespace acting as the field separator by default.

  • Given that a field-based solution isn't possible here - the fields have fixed width, so there's no separator (-t, --field-separator) you can specify - you'd have to use -k 1.40 to achieve column-based sorting, which is (a) far from obvious and (b) is the equivalent of passing { $_.substring(39) } to Sort-Object's -Property parameter, as in js2010's answer.


winget list | Sort-Object -Property Id

While -Property Id would indeed be wonderful if it worked, it cannot be expected to work with the text representations that the external program winget.exe outputs: what PowerShell then sees in the pipeline are strings, about whose content nothing is known, so they can't be expected to have an .Id property.

Should the functionality provided by winget.exe ever be exposed in a PowerShell-native way,[1] i.e. via cmdlets, they would indeed produce (non-string) objects with properties that would allow you to use Sort-Object -Property Id.


Dealing with winget.exe directly comes with the following challenges, owing to its nonstandard behavior (see also the bottom section):

  • It doesn't respect the current console's code page and instead invariably outputs UTF-8-encoded output.

    • To compensate for that, [Console]::OutputEncoding must (temporarily) be set to [System.Text.UTF8Encoding]::new()
  • It doesn't modify its progress-display behavior based on whether its stdout stream is connected directly to a console (terminal) or not; that is, it should suppress progress information when its output is being captured or redirected, but it currently isn't.

    • To compensate for that, the initial output lines that are the result of winget.exe's progress display must be filtered out.

Thus, an adapted version of js2010's answer would look like this:

# Make PowerShell interpret winget.exe's output as UTF-8.
# You may want to restore the original [Console]::OutputEncoding afterwards.
[Console]::OutputEncoding = [System.Text.UTF8Encoding]::new() 

(winget list) -match '^\p{L}' | # filter out progress-display and header-separator lines
  Select-Object -Skip 1 |       # skip the header line
  Sort-Object { $_.Substring(39) }

Parsing winget.exe list output into objects:

The textual output from winget.exe list reveals an inherent limitation that PowerShell-native commands with their separation of data output from its presentation do not suffer from: truncating property values with represents omission of information that cannot be recovered.

Thus, the following solution is limited by whatever information is present in winget.exe's textual output.

Assuming that helper function ConvertFrom-FixedColumnTable (source code below) is already defined, you can use it to transform the fixed-with-column textual output into objects ([pscustomobject] instances) whose properties correspond to the table's columns, which then allows you to sort by properties (columns), and generally enables OOP processing of the output.

[Console]::OutputEncoding = [System.Text.UTF8Encoding]::new() 

(winget list) -match '^(\p{L}|-)' | # filter out progress-display lines
  ConvertFrom-FixedColumnTable |    # parse output into objects
  Sort-Object Id |                  # sort by the ID property (column)
  Format-Table                      # display the objects in tabular format

ConvertFrom-FixedColumnTable source code:

# Note:
#  * Accepts input only via the pipeline, either line by line, 
#    or as a single, multi-line string.
#  * The input is assumed to have a header line whose column names
#    mark the start of each field
#    * Column names are assumed to be *single words* (must not contain spaces).
#  * The header line is assumed to be followed by a separator line
#    (its format doesn't matter).
function ConvertFrom-FixedColumnTable {
  [CmdletBinding()]
  param(
    [Parameter(ValueFromPipeline)] [string] $InputObject
  )
  
  begin {
    Set-StrictMode -Version 1
    $lineNdx = 0
  }
  
  process {
    $lines = 
      if ($InputObject.Contains("`n")) { $InputObject.TrimEnd("`r", "`n") -split '\r?\n' }
      else { $InputObject }
    foreach ($line in $lines) {
      ++$lineNdx
      if ($lineNdx -eq 1) { 
        # header line
        $headerLine = $line 
      }
      elseif ($lineNdx -eq 2) { 
        # separator line
        # Get the indices where the fields start.
        $fieldStartIndices = [regex]::Matches($headerLine, '\b\S').Index
        # Calculate the field lengths.
        $fieldLengths = foreach ($i in 1..($fieldStartIndices.Count-1)) { 
          $fieldStartIndices[$i] - $fieldStartIndices[$i - 1] - 1
        }
        # Get the column names
        $colNames = foreach ($i in 0..($fieldStartIndices.Count-1)) {
          if ($i -eq $fieldStartIndices.Count-1) {
            $headerLine.Substring($fieldStartIndices[$i]).Trim()
          } else {
            $headerLine.Substring($fieldStartIndices[$i], $fieldLengths[$i]).Trim()
          }
        } 
      }
      else {
        # data line
        $oht = [ordered] @{} # ordered helper hashtable for object constructions.
        $i = 0
        foreach ($colName in $colNames) {
          $oht[$colName] = 
            if ($fieldStartIndices[$i] -lt $line.Length) {
              if ($fieldLengths[$i] -and $fieldStartIndices[$i] + $fieldLengths[$i] -le $line.Length) {
                $line.Substring($fieldStartIndices[$i], $fieldLengths[$i]).Trim()
              }
              else {
                $line.Substring($fieldStartIndices[$i]).Trim()
              }
            }
          ++$i
        }
        # Convert the helper hashable to an object and output it.
        [pscustomobject] $oht
      }
    }
  }
  
}

Optional reading: potential winget.exe improvements:

  • The fact that winget.exe doesn't honor the console code page (as reported by chcp / [Console]::OutputEncoding) and instead invariably outputs UTF-8 is problematic, but somewhat justifiable nowadays, given that UTF-8 has become the most widely used character encoding, across all platforms, and is capable of encoding all Unicode characters, whereas the legacy Windows code pages are limited to 256 characters. Other utilities have made a similar decision, notably node.exe, the NodeJS CLI (Python is non-standard too, but has chosen the legacy ANSI code page as its default, though can be configured to use UTF-8).

    • In fact, it is the use of UTF-8 that enables use of (the horizontal ellipsis character U+2026) in the output, which is a space-efficient way to indicate omission of data (the ASCII alternative would be to use ..., i.e. three (.) characters.

    • winget.exe's encoding behavior isn't a problem if you've configured your (Windows 10 and above) system to use UTF-8 system-wide, which, however, has far-reaching consequences - see this answer.

    • Now that PowerShell (Core) itself consistently defaults to UTF-8, you could argue that even if the system as a whole doesn't use UTF-8 PowerShell console windows should - see GitHub issue #7233.

  • winget.exe should test whether its stdout stream is connected to a console (terminal) and only then output progress information, so as to avoid polluting its stdout data output.

  • The currently unavoidable truncation of column values that exceed the fixed column width could be avoided with an opt-in mechanism to provide output in a structured text format that is suitable for programmatic processing, such as CSV, similar to what the (now deprecated) wmic.exe utility has always offered with its /format option.

    • As noted, if in the future PowerShell cmdlets that provide the same functionality as winget.exe are made available, the problem wouldn't even arise there, given PowerShell's fundamental separation between (strongly typed) data and its - selectable - for-display representation.

[1] WinGet for PackageManagement is an example of a third-party module aimed at that.

Monochord answered 3/11, 2022 at 3:2 Comment(2)
Great details there, but why doesn't winget adhere to modern powershell native behavior? After all, it ships by default in every windows version, AFAIK. And fix the hugely annoying ellipsis truncation issue, that become more apparent the more variations of long named tools are used, mostly by Microsoft...Drawee
@not2qubit, please see my update (new bottom section).Monochord
W
7

It outputs text, not an object with properties like "Id". This program's output isn't very smart. It looks like it outputs some special characters as well like (U+2026 HORIZONTAL ELLIPSIS). The first thing that occurs to me is to cut off the first 39 characters and then sort it by column 40 onward, where Id starts. That should be like sort -k in unix. I believe a powershell version of winget is coming in the future. Replacing non-ascii with spaces and skipping the first 4 lines.

# or -creplace '\P{IsBasicLatin}'
(winget list) -replace '[^ -~]',' ' | select-object -skip 4 | 
  sort-object { $_.substring(39) }

Python 3.10.0 (64-bit)                 {21b42743-c8f9-49d7-b8b6-b5855317c7ed} 3.10.150.0
paint.net                              {28718A56-50EF-4867-B4C8-0860228B5EC9} 4.3.8
Microsoft Support and Recovery Assist  0527a644a4ddd31d                       17.0.7018.4
Name                                   Id                                     Version                 Available  Source
ScreenToGif                            NickeManarin.ScreenToGif               2.37.1                             winget
Microsoft .NET SDK 6.0.402 (x64)       Microsoft.DotNet.SDK.6                 6.0.402                            winget
3D Viewer                              Microsoft.Microsoft3DViewer_8wekyb3d8  7.2010.15012.0
Microsoft Sticky Notes                 Microsoft.MicrosoftStickyNotes_8wekyb  3.8.8.0
Paint 3D                               Microsoft.MSPaint_8wekyb3d8bbwe        6.2009.30067.0
Microsoft Visual C++ 2015-2019 Redist  Microsoft.VCRedist.2015+.x64           14.28.29325.2           14.34.318  winget

Trying out the Cobalt module that uses Crescendo to parse Winget. There's no name property, and version is just a string (apparently these things are more of a challenge). There's a lot of guid's at the top.

install-module cobalt -scope currentuser
get-wingetpackage | sort id

ID                                      Version             Available Source
--                                      -------             --------- ------
{04F3299A-F322-45A6-8281-046777B9C736}  21.0.3
{0E8670B8-3965-4930-ADA6-570348B67153}  11.0.2100.60
{0EDB70B6-EEA7-413B-BBC4-89E2CD36EFDE}  11.5.18
#...
7zip.7zip                               21.07               22.01     winget
Acrylic Suite
Acrylic Wi-Fi Home
Warm answered 1/11, 2022 at 14:55 Comment(13)
First column isn't necessarily 39 characters in width. It depends on the size of the console window. A better approach would be to determine column width from the header line.Unsling
@Unsling That isn't my experience. Plus in the beginning there's 2 blank lines with 3 or 4 backspaces in them. I know sometimes the number of columns vary.Warm
I can reproduce this on Win 10 with both cmd and powershell consoles. Example row from PowerShell console: Visual Studio Community 2022 Microsoft.VisualStudio.2022.Community 17.2.6 17.3.6 winget -> 1st column of 89 charactersUnsling
Yes, I don't know why it insist setting the width of the 1st column to 40 chars (UTF-8 ... ellipsis included). Madness. @zett42, what powershell version are you using and what is the charset (codepage) and OutputEncoding for that?Drawee
Parsing the output of this program is a unique challenge indeed.Warm
I think when you pipe it to something it stays 40 characters, although there's some encoding issues.Warm
Interesting, winget uses a fixed column width only when redirected. So substring(39) actually seems to work. I'd only suggest to filter the header lines from the sorted output now. winget | select -skip 4 works for me.Unsling
@Unsling ok, thank you, I'm also replacing non-ascii with spacesWarm
Interestingly, I just noticed I got an exception with your 1st command: Sort-Object: Exception calling "Substring" with "1" argument(s): "startIndex cannot be larger than length of string. (Parameter 'startIndex')", even though it worked. And it is also really weird that you have to skip 4 when there are only 2 lines shown. Does windows EOL make \n and \r both count?Drawee
@Drawee winget outputs 2 blank lines first with backspaces and other characters.Warm
Weird, now it works for me only with substring(38).Unsling
@Unsling It should work with a column of spaces in the front.Warm
No, your command trims the 1st character from the ID column for me. Possibly depending on localization of the OS?Unsling
U
3

Here is my take on the problem to avoid hardcoded position of the ID column. At least on my german-localized system, the column is one char off to the left. Search for the ID word in the header row to determine how much to chop off for sorting.

# Strip two blank lines and split into header string and items array 
$wgHdr, $null, $wgItems = winget list | Select-Object -skip 2

# Get the position of the 'ID' column.
$idPos = [regex]::Match( $wgHdr,'\bID\b' ).Index

# Sort beginning at the position of the 'ID' column
$wgItems | Sort-Object { $_.Substring( $idPos ) }
Unsling answered 1/11, 2022 at 20:49 Comment(0)
T
3

There's a request in the offical repo for a proper PS module, which has partially been completed, except it has to be built from source:

https://github.com/microsoft/winget-cli/tree/master/src/PowerShell/Microsoft.WinGet.Client

Although someone has packaged it using Scoop (another package manager).

https://github.com/microsoft/winget-cli/issues/221#issuecomment-1403206756

For now, it can be installed using the winget-ps Scoop package:

  1. Install Scoop: irm get.scoop.sh | iex
  2. Install winget-ps: scoop install winget-ps
  3. Reload PS or import module: Import-Module Microsoft.WinGet.Client

However, there appears to be a bug in version 1.4.10173 of winget-ps, although I wouldn't rule out another issue with my troubled Windows 11 (22H2) environment. It was me.

Thematic answered 11/2, 2023 at 11:8 Comment(1)
What's the manual way to install, without using scoop? There's an XML file in there as well, is it needed?Drawee
A
2

For a few months now, we can use the official Microsoft.WinGet.Client PowerShell Module to get proper output that can be handled by other PowerSehll commands.

Find-WinGetPackage notepad++

Name         Id                   Version Source
----         --                   ------- ------
Notepad++    Notepad++.Notepad++  8.6.9   winget
Notepad Next dail8859.NotepadNext 0.8     winget
Find-WinGetPackage notepad++|Sort-Object -Property Version

Name         Id                   Version Source
----         --                   ------- ------
Notepad Next dail8859.NotepadNext 0.8     winget
Notepad++    Notepad++.Notepad++  8.6.9   winget

The solution to @not2qubit question would be:

Get-WinGetPackage | Sort-Object -Property Id
Abisha answered 21/8 at 9:26 Comment(1)
Thanks for good option, but it's ridiculous that winget itself cannot handle this without a hacky wrapper.Drawee
E
1

The solution from mkelement0 worked great except for one thing.

It wasn't showing the 7-Zip (7zip.7zip) application (for instance). It had to do with the regex match and the hyphen in the application name.

(winget list) -match '^(\p{L}|-)' | # filter out progress-display lines
  ConvertFrom-FixedColumnTable |    # parse output into objects
  Sort-Object Id |                  # sort by the ID property (column)
  Format-Table                      # display the objects in tabular format 

Not being one of the chosen few who deeply understand regex, I replaced that portion with a 2nd helper function, winget_outclean, to filter out the progress lines. I also noticed a few rows in the Version column look like "> 5.16.1.22523". I couldn't find documentation on what the > means, so I removed it with the select-object expression.

 winget list | winget_outclean |    # filter out progress-display lines
  ConvertFrom-FixedColumnTable |    # parse output into objects
  Sort-Object Id               |    # sort by the ID property (column)
  Select-Object Name,Id,@{N='Version';E={$_.Version.Replace("> ","")}},Available,Source # Version fixup

Here's the helper function. It basically ignores preamble lines until it sees the header line 'Name....'

function winget_outclean () {
  [CmdletBinding()]
  param (
    [Parameter(ValueFromPipeline)]
    [String[]]$lines
  )
if ($input.Count -gt 0) { $lines = $PSBoundParameters['Value'] = $input }
  $bInPreamble = $true
  foreach ($line in $lines) {
    if ($bInPreamble){
      if ($line -like "Name*") {
        $bInPreamble = $false
      }
    }
    if (-not $bInPreamble) {
        Write-Output $line
    }
  }
}
Elin answered 1/10, 2023 at 11:22 Comment(1)
I can't find any function called ConvertFrom-FixedColumnTable. Where did you get that from?Drawee
P
0

In case that my edit on post of mklement0 may be removed; I am adding my fix (at $colNames creation) on ConvertFrom-FixedColumnTable source code at here:

Update: I removed try-catch and added a if-else check for that.

Update: The source blog is fixed at last. Tou can get code for ConvertFrom-FixedColumnTable at here. You may also take a look to my winget_list_OrderBy function at here (which uses ConvertFrom-FixedColumnTable in it) to take it a step forward.

Printery answered 27/1, 2023 at 18:32 Comment(0)
L
0

The winget_outclean function by Jason Simotas above solves the problem of hyphenated package names, but there's one more hurdle if this is the first time you've run winget on the PC: the source agreement approval prompt. If you don't approve this, your unattended powershell script can hang and it could be difficult to determine why. The winget line now reads like this:

winget list --accept-source-agreements --disable-interactivity | winget_outclean | # filter out progress-display lines

Also, this alternative code outputs to a CSV file sorted by Name and includes the PC serial number and report date:

$MachineName = (gwmi win32_bios).serialnumber
$ReportDate = [DateTime]::Now.ToString("yyyyMMddHHmmss")
$CsvFile = "C:\SCRIPTS\$($MachineName).$($ReportDate).wingetlist.csv"

winget list --accept-source-agreements --disable-interactivity | winget_outclean | # filter out progress-display lines
  ConvertFrom-FixedColumnTable |    # parse output into objects
  Sort-Object Name |                  # sort by the Name property (column)
  Select-Object @{Label="PC"; Expression={$MachineName}}, @{Label="RptDate"; Expression={$ReportDate}}, * |
  Export-Csv -Path $CsvFile -NoTypeInformation
Lifeguard answered 26/3 at 19:1 Comment(0)

© 2022 - 2024 — McMap. All rights reserved.