Different length arrays to one CSV
Asked Answered
D

3

3

If you have multiple arrays of different length, how can you export these to a single csv in powershell?

Array1 = 1,2,3 
Array2 = Bob,smithy,Alex,Jeremy 
Array3 = yes,no

Output CSV

Number  Name  Valid 
———————————————————  
1      Bob    Yes  
2      Smithy no  
3      Alex
       Jeremy

Basically each array would be in its own header column.

Tried lines like

Array1 | Select-Object Number | export-csv -Path C:\Path

This works for singular arrays to singular csv files

But if I try

Array1, Array2, Array3 | Select-Object Number, Name, Valid | export-csv -Path C:\Path

I just get the header names and no values in the columns

Damales answered 7/9, 2022 at 18:56 Comment(0)
E
7

One way to do it is with a for loop.

$Array1 = 1, 2, 3
$Array2 = 'Joe Bloggs', 'John Doe', 'Jane Doe'
$Array3 = 'Yes', 'No'

$export = for($i = 0; $i -lt [Linq.Enumerable]::Max([int[]] ($Array1.Count, $Array2.Count, $Array3.Count)); $i++) {
    [pscustomobject]@{
        Number = $Array1[$i]
        Name   = $Array2[$i]
        Valid  = $Array3[$i]
    }
}
$export | Export-Csv path\to\csv.csv -NoTypeInformation

Another example using a function, the logic is more or less the same except that there is more overhead involved, since this function can handle an indefinite amount of arrays coming from the pipeline.

function Join-Array {
    [CmdletBinding()]
    param(
        [parameter(ValueFromPipeline, Mandatory)]
        [object[]] $InputObject,

        [parameter(Mandatory, Position = 0)]
        [string[]] $Columns
    )

    begin {
        $inputDict = [ordered]@{}
        $index = 0
    }
    process {
        try {
            if ($MyInvocation.ExpectingInput) {
                return $inputDict.Add($Columns[$index++], $InputObject)

            }

            foreach ($item in $InputObject) {
                $inputDict.Add($Columns[$index++], $item)
            }
        }
        catch {
            if ($_.Exception.InnerException -is [ArgumentNullException]) {
                $errorRecord = [Management.Automation.ErrorRecord]::new(
                    [Exception] 'Different count between input arrays and Columns.',
                    'InputArrayLengthMismatch',
                    [Management.Automation.ErrorCategory]::InvalidOperation,
                    $InputObject
                )
                $PSCmdlet.ThrowTerminatingError($errorRecord)
            }
            $PSCmdlet.ThrowTerminatingError($_)
        }
    }
    end {
        foreach ($pair in $inputDict.GetEnumerator()) {
            $count = $pair.Value.Count
            if ($count -gt $max) {
                $max = $count
            }
        }

        for ($i = 0; $i -lt $max; $i++) {
            $out = [ordered]@{}
            foreach ($column in $inputDict.PSBase.Keys) {
                $out[$column] = $inputDict[$column][$i]
            }
            [pscustomobject] $out
        }
    }
}

The usage would be pretty easy, the arrays to join / zip can be passed through the pipeline and the desired column names defined positionally:

$Array1 = 1, 2, 3
$Array2 = 'Joe Bloggs', 'John Doe', 'Jane Doe'
$Array3 = 'Yes', 'No'
$Array4 = 'hello', 'world', 123, 456

$Array1, $Array2, $Array3, $Array4 | Join-Array Number, Name, Valid, Test |
    Export-Csv path\to\csv.csv -NoTypeInformation
Epicotyl answered 7/9, 2022 at 19:8 Comment(1)
I did not know you could assign a for loop to a variable in that way, though in hindsight it makes some sense if each iteration just leaves a value on the pipeline. I'll definitely be using this technique more in the future.Stilted
C
2

For quite some time I've been maintaining a Join-Object script/Join-Object Module (see also: In PowerShell, what's the best way to join two tables into one?).
Its main purpose is joining tables (aka lists of objects) based on a related column (aka property) defined by the -on parameter using a user (aka scripter) friendly syntax which as many joining options as possible. This has led to a few implicit features that could be used in your specific request:

  • If there is no relation defined (the -on parameter is omitted), it is assumed that you want to do a side-by-side join of the tables (aka lists)
    • If the tables (or lists) are not equal in size, a common (inner) join will end when either lists ends. A full join (e.g. FullJoin-Object alias FullJoin or a left - or right join) at the other hand, will join the complete respective lists
  • An array of scalars (e.g. strings rather than custom objects) is joined as being a list of objects where the (default) column (aka property) name is Value
    • If both sides contain a list of scalars, the output will be a [Collections.ObjectModel.Collection[psobject]] list for each left and right item, containing those items.
  • Join commands can be chained: ... | Join ... | Join ...
  • If the columns (aka properties) provided in the tables (aka lists) overlap, they will be merged in an array ( <property name> = <left property value>, <right property value>) by default, which items could also be assigned to a specific name (prefix) using the -Name parameter. This parameter can be used with every join command in the chain.

All together this means that you could implement your task with the following command line:

$Array1 | FullJoin $Array2 | FullJoin $Array3 -Name Number, Name, Valid
Number Name       Valid
------ ----       -----
     1 Joe Bloggs Yes
     2 John Doe   No
     3 Jane Doe

Update (2023-22-06)

Since version 3.8.1 it is possible to dynamically add arrays (starting from an empty array initiator), see also issue #42. Taking this question as an example:

As a new transposed jagged array:

$dummy = @(("a","b","c"),("1","2","3"),("John","Jane","Smith"))
$dummy |Foreach-Object { $a = @() } { $a = $a | FullJoin $_ }

$a.ForEach{ $_ -Join '|' }

a|1|John
b|2|Jane
c|3|Smith

converted to an object collection:

$a | FullJoin @() -Name foo, bar, baz

foo bar baz
--- --- ---
a   1   John
b   2   Jane
c   3   Smith

dynamically naming:

$dummy |Foreach-Object { $a = @(); $n = 1 } { $a = $a | FullJoin $_ -Name "Name$(($n++))" }
$a

Name1 Name2 Name3
----- ----- -----
a     1     John
b     2     Jane
c     3     Smith
Clabo answered 8/9, 2022 at 6:36 Comment(0)
S
0

You need to restructure the data to pivot it into one array (presented here in pseudo-json):

[ 
  {"1", "Bob", "Yes"}, 
  {"2", "Smithy", "no"}, 
  {"3", "Alex", ""}, 
  {"", "Jeremy", ""} 
]

Note how the missing fields still appear with empty placeholders.

Once you've pivoted your data this way, writing it to a CSV file is trivial, as are many other tasks (this is the better way to structure things in the first place).

Stilted answered 7/9, 2022 at 19:8 Comment(1)
If each array is being created from importing data in other csv how would you make one big array with multiple imports?Damales

© 2022 - 2025 — McMap. All rights reserved.