Powershell Selecting NoteProperty Type Objects From Object

Question

I am working with deeply nested JSON and, after convertfrom-json, need to be able to traverse through various parts of the object which the convertfrom-json cmdlet generates.

I have no way of knowing in advance what property names may or may not be inside the object, as far as I can tell, there are hundreds of different possible properties. Fortunately the one thing I am seeing that helps is that each of the properties I care about is of type "NoteProperty".

Here is an example:

TypeName: System.Management.Automation.PSCustomObject

Name               MemberType   Definition
----               ----------   ----------
Equals             Method       bool Equals(System.Object obj)
GetHashCode        Method       int GetHashCode()
GetType            Method       type GetType()
ToString           Method       string ToString()
definition         NoteProperty System.Management.Automation.PSCustomObject definition=@{$schema=https://schema.management.azure.com/providers/Microsof... 
integrationAccount NoteProperty System.Management.Automation.PSCustomObject integrationAccount=@{id=[parameters('integrationAccounts_xxx_integration... 
parameters         NoteProperty System.Management.Automation.PSCustomObject parameters=@{$connections=}
state              NoteProperty string state=Enabled

So I thought it would be simple to create a function which would select only the objects, for the level currently being processed, which are of 'MemberType' 'NoteProperty'.

I have tried piping the object to:

where-object { $_.MemberType -eq "NoteProperty" }

Nope.

I have tried select-object in various forms too but can't seem to select just what I need. I found an old article from the Scripting guys about using Labels and Expressions - but that seems like overkill, no? Can someone point me to simple way to select just the NoteProperty items?

Thanks!

Shinleaf

take a look at the hidden .PSObject property of most powershell objects. it contains a property named .Properties that will give you all the props for the object ... and you can iterate thru them. if you need to dig into sub-properties, then you may find it easier to use regex on the raw JSON ... [sigh ...] – Caudex

Another approach would be to use Get-Member -MemberType NoteProperty to get the property names. I think PSObject.Properties makes more sense, though. – Goodrich

@Caudex Thank you so much! Yes, that is a great solution and thanks to your suggestion I am up and running again. Thank you! – Shinleaf

@Shinleaf - kool! glad to have helped a little bit ... [grin] – Caudex

Answer 1

You could use the hidden .psobject.properties to iterate over the members.

$json = @'
{
  "users": [
    {
      "userId": 1,
      "firstName": "Krish",
      "lastName": "Lee",
      "phoneNumber": "123456",
      "emailAddress": "[email protected]"
    },
    {
      "userId": 2,
      "firstName": "racks",
      "lastName": "jacson",
      "phoneNumber": "123456",
      "emailAddress": "[email protected]"
    }
  ]
}
'@ | ConvertFrom-Json

$json | foreach {
    $_.psobject.properties | foreach {
        Write-Host Property Name: $_.name
        Write-Host Values: $_.value
    }
} 

You can keep going as needed.

$json | foreach {
    $_.psobject.properties | foreach {
        $_.value | foreach {
            $_.psobject.properties | foreach {
                write-host Property name: $_.name
                write-host Property value: $_.value
            }
        }
    }
}

Property name: userId
Property value: 1
Property name: firstName
Property value: Krish
Property name: lastName
Property value: Lee
Property name: phoneNumber
Property value: 123456
Property name: emailAddress
Property value: [email protected]
Property name: userId
Property value: 2
Property name: firstName
Property value: racks
Property name: lastName
Property value: jacson
Property name: phoneNumber
Property value: 123456
Property name: emailAddress
Property value: [email protected]

Sasser

Yes! This technique works out really well for me... some work to do to filter down on the output as I am recursively checking to see if the current $val is a PSCustomObject and it it is I am sending it through the function again to unpack everything in there... so a little messy - but it works! Thanks! – Shinleaf

The .PSObject is an intrinsic member and is documented here: about_intrinsic_members – Stealage

Answer 2

To complement Doug Maurer's helpful answer with a generalized solution:

The following snippet defines and calls function Get-LeafProperty, which recursively walks an object graph - such as returned by ConvertFrom-Json - and outputs all leaf property values, along with their name paths in the hierarchy.

# Define a walker function for object graphs:
# Get all leaf properties in a given object's hierarchy,
# namely properties of primitive and quasi-primitive types 
# (.NET primitive types, plus those that serialize to JSON as a single value).
# Output:
#  A flat collection of [pscustomobject] instances with .NamePath and .Value 
#  properties; e.g.:
#   [pscustomobject] @{ NamePath = 'results.users[0].userId'; Value = 1 }
function Get-LeafProperty {
  param([Parameter(ValueFromPipeline)] [object] $InputObject, [string] $NamePath)
  process {   
    if ($null -eq $InputObject -or $InputObject -is [DbNull] -or $InputObject.GetType().IsPrimitive -or $InputObject.GetType() -in [string], [datetime], [datetimeoffset], [decimal], [bigint]) {
      # A null-like value or a primitive / quasi-primitive type -> output.
      # Note: Returning a 2-element ValueTuple would result in better performance, both time- and space-wise:
      #      [ValueTuple]::Create($NamePath, $InputObject)
      [pscustomobject] @{ NamePath = $NamePath; Value = $InputObject }
    }
    elseif ($InputObject -is [System.Collections.IEnumerable] -and $InputObject -isnot [System.Collections.IDictionary]) {
      # A collection of sorts (other than a string or dictionary (hash table)), 
      # recurse on its elements.
      $i = 0
      foreach ($o in $InputObject) { Get-LeafProperty $o ($NamePath + '[' + $i++ + ']') }
    }
    else { 
      # A non-quasi-primitive scalar object or a dictionary:
      # enumerate its properties / entries.
      $props = if ($InputObject -is [System.Collections.IDictionary]) { $InputObject.GetEnumerator() } else { $InputObject.psobject.properties }
      $sep = '.' * ($NamePath -ne '')
      foreach ($p in $props) {
        Get-LeafProperty $p.Value ($NamePath + $sep + $p.Name)
      }
    }
  }
}

Example use:

# Parse sample JSON with multiple hierarchy levels into a [pscustomobject]
# graph using ConvertFrom-Json.
$objectGraphFromJson = @'
{
  "results": {
      "users": [
          {
              "userId": 1,
              "emailAddress": "[email protected]",
              "attributes": {
                  "height": 165,
                  "weight": 60
              }
          },
          {
              "userId": 2,
              "emailAddress": "[email protected]",
              "attributes": {
                  "height": 180,
                  "weight": 72
              }
          }
      ]
  }
}
'@ | ConvertFrom-Json

# Get all leaf properties.
Get-LeafProperty $objectGraphFromJson

The above yields:

NamePath                                          Value
--------                                          -----
results.users[0].userId                               1
results.users[0].emailAddress      [email protected]
results.users[0].attributes.height                  165
results.users[0].attributes.weight                   60
results.users[1].userId                               2
results.users[1].emailAddress      [email protected]
results.users[1].attributes.height                  180
results.users[1].attributes.weight                   72

Niggard

I use your code to compare 2 json files and merge the result. Is there a way to convert this back to a json? Ofc with the original structure. (ConvertTo-Json) alone will not do. – Uncritical

That's an interesting use case, @njefsky, and I encourage you to ask a new question focused on that. One conceptual problem that comes to mind is: say you have a result with path results.users[4].attributes.weight, but not also for array indices 0 through 3, how would you handle that? – Niggard

I have the merge part covered, in your example I make sure the order and number of the users are the same. The thing I do not seem able to solve is how to translate the path back to a json again. – Uncritical

Answer 3

To further complement mklement0's helpful answer with an even more general (custom) solution, this ObjectGraphTools module lets you explore all the nodes in an object-graph with e.g. the Get-ChildNode cmdlet:

$objectGraphFromJson | Get-ChildNode -Recurse -Leaf

Path                               Name         Depth                Value
----                               ----         -----                -----
results.users[0].userId            userId           4                    1
results.users[0].emailAddress      emailAddress     4 [email protected]
results.users[0].attributes.height height           5                  165
results.users[0].attributes.weight weight           5                   60
results.users[1].userId            userId           4                    2
results.users[1].emailAddress      emailAddress     4 [email protected]
results.users[1].attributes.height height           5                  180
results.users[1].attributes.weight weight           5                   72

Access a node by its dot-notation path using the Get-Node cmdlet to e.g. easily change its value (whether or not it is a [PSCustomObject] or any type of Dictionary):

$objectGraphFromJson | Get-Node results.users[0].attributes.height

Path                               Name   Depth Value
----                               ----   ----- -----
results.users[0].attributes.height height     5   165

or using Extended dot-notation (Xdn):

$objectGraphFromJson | Get-Node  ~userId=2..emailAddress

Path                          Name         Depth Value
----                          ----         ----- -----
results.users[1].emailAddress emailAddress     4 [email protected]

And do things along with comparing object-graphs using Compare-ObjectGraph etc.

Potation

Answer 4

Another way you could do this is to get your results into some select expressions and then sort on that.

In yours, you have integrationAccount and parameters as note properties, so if these were in a pscustomobject called $results

$results = $results | Select @{l="integrationAccount ";e={$_.integrationAccount }},@{l="parameters ";e={$_.parameters }}

Then you'll be able to do a Select and/or Where-Object on that.

Episcopacy