Unpivot an Excel matrix/pivot-table?
Asked Answered
F

6

3

Is there a quick way to "unpivot" an Excel matrix/pivot-table (in Excel or elsewhere), without writing macros or other code ?
Again, I can write code (C# or VBA or whatever) that does that myselfs.
I want to know if it is possible to do it without code, quickly ?

E.g. I need to convert this permission matrix (given as Excel-table/matrix)

Pivoted

into this half-normalized table (so I can insert it into a SQL database):

Unpivoted

e.g. in SQL I could do it like this:

CREATE TABLE dbo.T_DocumentMatrix
(
    [Function] [varchar](255) NULL,
    [GROUP-Admin] [varchar](255) NULL,
    [GROUP-SuperUser] [varchar](255) NULL,
    [GROUP-Manager] [varchar](255) NULL,
    [GROUP-OLAP] [varchar](255) NULL,
    [GROUP-1] [varchar](255) NULL,
    [GROUP-2] [varchar](255) NULL,
    [GROUP-3] [varchar](255) NULL,
    [GROUP-4] [varchar](255) NULL,
    [GROUP-5] [varchar](255) NULL,
    [GROUP-6] [varchar](255) NULL,
    [GROUP-7] [varchar](255) NULL,
    [GROUP-8] [varchar](255) NULL,
    [Externals] [varchar](255) NULL
); 

copy-paste the data from excel, and then

SELECT * 
FROM 
(
    SELECT 
         [Function]
        ,[GROUP-Admin]
        ,[GROUP-SuperUser]
        ,[GROUP-Manager]
        ,[GROUP-OLAP]
        ,[GROUP-1]
        ,[GROUP-2]
        ,[GROUP-3]
        ,[GROUP-4]
        ,[GROUP-5]
        ,[GROUP-6]
        ,[GROUP-7]
        ,[GROUP-8]
        ,[Externals]
    FROM T_DocumentMatrix
) AS p
UNPIVOT
(
    Rights FOR GroupName IN 
    (
         [GROUP-Admin]
        ,[GROUP-SuperUser]
        ,[GROUP-Manager]
        ,[GROUP-OLAP]
        ,[GROUP-1]
        ,[GROUP-2]
        ,[GROUP-3]
        ,[GROUP-4]
        ,[GROUP-5]
        ,[GROUP-6]
        ,[GROUP-7]
        ,[GROUP-8]
        ,[Externals]
    )
) AS unpvt
;

However, that requires I change the table-create script and the unpivot-script for every change in groups...

Fowliang answered 20/8, 2015 at 10:1 Comment(2)
It is possible to do it pretty quick: superuser.com/a/78464 (but you will probably need to replace * with 1 first).Claudianus
@user3964075: Actually you don't need to replace *, it will work fine as string. Found that post before, but didn't work for me, because I couldn't start the pivot wizard. Found the answer on youtube.Fowliang
F
4

Oh, well, it's a little complicated. One of the problems is, the wizard-callup shortcuts don't work in non-english versions of excels (damn, at home I would have the English version, but here at work...)

Here's a good video: https://www.youtube.com/watch?v=pUXJLzqlEPk

But youtube videos can be deleted, so to make it a solid SO answer:

First, you need to go to "Options", and add the menuband-item "Pivot table and PivotChart Wizard".

Options

Wizard

Create a multiple consolidation pivot table Multiple-consolid

and use the custom variant
custom variant

and select the range, and in new work sheet range

then delete rows and columns fields

delete rows and columns

Double click on the NUMBER (54 in the picture)

values consolidtions

and excel will give you the halfway normalized data.

halfway

Fowliang answered 20/8, 2015 at 11:10 Comment(0)
S
4

While this is a really old question and Stefan found an enlightened answer back in the day, it could be worth a revisit. I have run into a need for such a codeless, dynamic Unpivot method myself and a Google search brought me here. Yes, Power Query does the job, but that is not exactly codeless in that there is a scripted background solution running in Power BI, it requires the user to Refresh the data (so, it is not automatic within the workbook) and it will not run on Excel for Mac (tmk).

The following is an approach based on Dynamic Arrays and using the LET function, so it will require Excel 2016 or Microsoft 365.

Let's say that Stefan's data are in cells A1 to N8. In Power Query parlance, we would say that Stefan wants to "Unpivot B1:N8 By A1:A8".

NB: The following approach would also accept multiple columns for By, e.g. you could have a need to "Unpivot D1:N8 By A1:C8".

=LET( unPivMatrix, B1:N8,
      byMatrix, A1:A8,
       upC, COLUMNS( unPivMatrix ),
       byC, COLUMNS( byMatrix ),
       dmxR, MIN( ROWS( unPivMatrix ), ROWS( byMatrix ) ) - 1,
       upCells, dmxR * upC,
       upSeq, SEQUENCE( upCells,, 0 ),
       upHdr, INDEX( INDEX( unPivMatrix, 1, ),  1,  SEQUENCE( upC ) ),
       upBody, INDEX( unPivMatrix,  SEQUENCE( dmxR ) + 1,  SEQUENCE( 1, upC ) ),
       byBody, INDEX( byMatrix,  SEQUENCE( dmxR ) + 1,  SEQUENCE( 1, byC ) ),
       attr, INDEX( upHdr, MOD( upSeq, upC ) + 1 ),
       mux, INDEX( upBody, upSeq/upC + 1, MOD( upSeq, upC ) + 1 ),
       demux, IFERROR( INDEX(
                             IFERROR( INDEX( byBody,
                                             IFERROR( INT( SEQUENCE( upCells, byC,0 )/byC/upC ) + 1, MOD( upSeq, upC ) + 1 ),
                                                      SEQUENCE( 1, byC + 1 ) ),
                                       attr ),
                             upSeq + 1, SEQUENCE( 1, byC + 2 ) ),
                        mux ),
       demux
     )

How this works - Reading the inputs

The inputs are the range that you want to unpivot that I called unPivMatrix B1:N8 (which can be of any dimensions you need) and the columns that you want to unpivot them by that I called byMatrix A1:A8.

As a rule, the rows of the byMatrix need to be the same as the unPivMatrix, so you must start with A1 and not A2. I decided to take this convention because their might be header in A1 and with some small mods, this formula could produce exactly the same output as Power Query, but that is not what Stefan asked for.

The formula first calculates the number of columns in each matrix: upC unpivot columns and byC by columns. From these it calculates dmxR (demultiplexed rows): how many rows of values will be delivered in the unpivot by taking the minimum of the rows of unPivMatrix and byMatrix and subtracting 1 because unPivMatrix has a header. Taking the MIN is an error prevention step in the event that the inputs have a different number of rows, which is an error by definition.

These values are used to create shaping variables that will be used later to form the output. upCells is the number of values that will be unpivoted and is used to generate an index pattern called upSeq that counts off the number of values from 0 which will used inside of the INDEX functions later. We start with 0 because upSeq will be modulated to form correct indexation of the inputs and outputs.

Now we will break down the parts of the matrices so that we can multiplex them. The parts look like this:

enter image description here

There is an unpivot header (upHdr) that contains the identifier of the value data (in Stefan's case, GROUP-Admin, GROUP-SuperUser, etc.). These will be multiplexed into a column that will later be placed next to each value that is unpivoted. upHdr is created by putting the whole unPivMatrix into an INDEX function and reading Row 1 and all of the columns. I nested that INDEX function into another INDEX that reshapes the horizontal array to a vertical array using a vertical SEQUENCE of size upC.

upBody contains the values that we want to unpivot via multiplexing. It is created by putting the upMatrix into an INDEX and shaping it by the number of rows that will be output SEQUENCE( dmxR ) + 1 § against each of the columns to be read SEQUENCE( upC ). byBody contains the data that will be multiplexed against each value in the upBody. It is created the same way as upBody.

§ - add 1 to skip the header row

How this works - Shaping and writing the outputs

The output will be shaped like this:

enter image description here

We now multiplex the upHdr into attr or attributes (using Power Query terminology) by putting upHdr into an INDEX and applying a modulated sequence based on upSeq that repeats every upC times (e.g. {1;2;3;4;5;6;7;8;1;2;...} ). Note: this is where starting with 0 in upSeq mattered. The output of attr would look like (in Stefan's case) {GROUP-Admin; GROUP-SuperUser, etc.}.

mux is the values (using Power Query terminology) that will be multiplexed against each attribute and byBody row. It is created by putting valBody into INDEX and then reshaping it into a multiplex pattern created by rows of

upSeq/upC + 1 which yields a row of {1,1,1,1,1,1,1,1,2,2,...}

and columns of

MOD( upSeq, upC ) + 1 which yields a column of {1;2;3;4;5;6;7;8;1;...}.

The output for mux will be the content of the unPivMatrix. In Stefan's case, that will be a little special because he is using * and blank as data. This formula will convert blanks into 0's. So, if that is a problem, you could either wrap mux into an IF( ISBLANK( mux ), "", mux ), but I am not adding that because I want a general unpivot as I am sure Stefan has long since moved on.

How this works - Demultiplexing the parts into the output

Now that the easy part is done, it is time to go after the hard stuff - bringing this all together into a single dynamic array. Putting multiple arrays together requires a trick and this trick has to be applied twice because, as you can see, we are bringing three tables together. The trick is like having an APPEND function like:

APPEND( APPEND( table1, table2 ), table3 ).

To combine two arrays, you put the first array into an INDEX and then reference cells that are outside of the array to force a #REF! error. For example, if I have a 3 x 2 array of letters A through F and I reference cell 3, 3, it will throw a reference error.

enter image description here

Now you can exploit the errors by replacing them with the table that you want to append by wrapping the INDEX in an IFERROR. It is kind of like:

IFERROR( INDEX( table1,
                SEQUENCE( table1.rows ),
                SEQUENCE( 1, table1.columns + table2.columns ) ),
         table2 )

In this sense, the formula above is the equivalent of APPEND( table1, table2 ) where APPEND is a row-wise append of two tables which is what we want. (NB: switch the sequence patterns and you can make a column-wise append.)

So, hopefully this explanation makes it clear what is happening in the final stages of a variable called demux which delivers the result. I named the result and then referenced it so that you can easily explore, modify or optimize the formula. So, demux is really like:

APPEND( byBody, APPEND( attr, mux ) )

I won't go into the mechanics of how this final stage works because this is already a really long answer, but a short summary is that this append is using the dimensions created by upCells, upC and byC to form the output.

I've tested this, but I've not performance streamlined it or brought it to #SwissEngineering standards.

Silvanus answered 15/5, 2021 at 10:54 Comment(10)
The answer is perfect and solves a big problem if I can get it working. But not able to use in Excel 2016. Excel does not recognize the LET function. Any clues?Masseter
Hey @Masseter - yes, in fact, after I posted this, I found out that LET and SEQUENCE are only in Excel 365 - not even 2019! Both are needed to pull this off. While there is a workaround for SEQUENCE that uses INDEX & ROW, the LET problem remains. LET can be eliminated by collapsing the formula into an unreadable mess, but the real issue will be calc time. LET allows your formula to be "DRY". The PowerQuery method posted here[#20542405 by Telwyn could work for you.Silvanus
Is it possible to apply this to retain multiple columns in the unpivoted data? For example, imagine the columns in the original are actually 2-dimensional, with month spanning 4 columns each that are repeated over 4*12=48 columns. I'd either need an unpivoted table with 4 values columns, or an unpivoted table with 2 attributes columnsCalabar
@Calabar - so, to understand, do you mean that the upHdr would be multiple columns that get repeated, row-wise? e.g. the columns are for each month the Pressure, Humidity, Temp, CO2. i.e. Jan Pressure, Jan Hum,... Feb Pressure, Feb Hum... and the rows might represent City, State. Then output would then have 4 values (P,h,T,C) for the values in image 2 above and the attr would have the months. If so, it would be like a python-pandas melt and, yes, it can be solved in a dynamic array formula.Silvanus
@Calabar - it reminds me a little of this OP: #68109601 but there would be multiple rows in the column headers, right?Silvanus
@Calabar - I made a prototype of the Place, Month, (P,h,T,C) unpivot. over the weekend. If that is what you are looking for, I can send it in Chat, but it needs explanation because I am not sure what would be the most logical way of defining the inputs. The Unpiv above is over-engineered so that it can be turned into a UDF when LAMBDA is released - thus the shaping, muxing and demuxing to accommodate malformed inputs.Silvanus
I would love to see it! I've never used chat. I'm currently using a cut and paste solution and hoping the original data doesn't exceed the rows I'm grabbingCalabar
@Calabar - OK. I tried chat for the first time and it will not give us much. It is really limited to text. So, I will work up a Question that is framed the way you did above and then put an answer to it based on what I think would be the best approach, but I will not mark it as the answer because there could be better approaches than what I am thinking and who knows what creativity we could uncover. Will work on that tonight (CET).Silvanus
@Calabar - ok, it is tested and done. I finished it last night pretty late. I need more time to structure it into a question. It is a LET with inputs like this: =LET( upValues, C3:N7, upHdr, C2:N2, blockSize, 4, upAttr, C1:N1, byBody, A3:B7, byHdr, A2:B2, attrTitle, "month", It might be better to make it behave like Pandas melt. I'm not sure how to frame the question around the answer and MELT seems to be a good definition.Silvanus
@Calabar - I finally got around to doing it. I actually had it working last night and then discovered that it breaks if you reorder the months, which is not ideal. Then it took a lot of time to document the question and answer. Going to bed now. Have a look when you have the time.Silvanus
K
3

Years later, Excel does have a faster way of doing this from one cell with a blend of formulas (or as described "quick"). It requires some newer functions such as ByRow and ByCol but these are scheduled to be available to everyone (someday).

To recap:

  • No VBA
  • Dynamic for multiple axes in both rows and columns data ranges
  • Can be converted to lambda (in desktop version)

With the below dataset pasted in cell A1, you could use this Lambda function to unpivot or flatten the data:

Starting Dataset

See sample file here

(cell A1) Jan Feb Mar Apr May
Actual Actual Actual Budget Budget
Disney Tickets Revenue 507 607 707 807 907
Disney Movies Costs 508 608 708 808 908
StarWars Promo Revenue 509 609 709 809 909
StarWars Vader Taxes 510 610 710 810 910
Marvel HR Costs 511 611 711 811 911

Stand Alone Formula

=LET(dataRng,D3:H7,  rowAxis,A3:C7, colAxis,D1:H2,
   iCol,COLUMN(INDEX(rowAxis,1,1)),   amountCol,TOCOL(dataRng),  totalCells,COUNTA(amountCol),
          HSTACK(
              INDEX(rowAxis,
                     INT(SEQUENCE(totalCells,1,0,1)/COLUMNS(dataRng))+1,
                     BYCOL(INDEX(rowAxis,1,),  LAMBDA(aCol,COLUMN(aCol) -iCol +1))),
              INDEX(colAxis,
                      SEQUENCE(1,ROWS(colAxis),1,1),
                      MOD(SEQUENCE(totalCells,1,0,1),COLUMNS(dataRng))+1),
               amountCol))
    

Lambda Formula

=LAMBDA(dataRng,rowAxis,colAxis,
   LET(iCol,COLUMN(INDEX(rowAxis,1,1)), amountCol,TOCOL(dataRng), totalCells,COUNTA(amountCol),
          HSTACK(
              INDEX(rowAxis,
                     INT(SEQUENCE(totalCells,1,0,1)/COLUMNS(dataRng))+1,
                     BYCOL(INDEX(rowAxis,1,),  LAMBDA(aCol,COLUMN(aCol) -iCol +1))),
              INDEX(colAxis,
                      SEQUENCE(1,ROWS(colAxis),1,1),
                      MOD(SEQUENCE(totalCells,1,0,1),COLUMNS(dataRng))+1),
               amountCol
                      )))(D3:H7,A3:C7,D1:H2)
    

enter image description here

Katha answered 1/4, 2023 at 3:16 Comment(6)
Thanks. Question on your Stand-Alone Formula: Instead of the BYCOL( ...LAMBDA construct, could it not be TRANSPOSE( SEQUENCE( COLUMNS( rowAxis ))) ? That would seem to be a simpler way of achieving the same result, but am wondering if I'm missing something.Sunlight
@Sunlight it was an April Fools joke, you were supposed to post your response then. Honestly, I am not sure if your modification is simpler but I'm sure you can appreciate the irony of your post offering nominal efficiency when your suggestion could have a formula reduction by 1/3. SEQUENCE(1,COLUMNS(rowAxis )) vs TRANSPOSE(SEQUENCE(COLUMNS(rowAxis ))) 👍🏻Katha
@JCM lies!! This is not a macro, but rather a customized excel formula. No need to reduce security settings to achieve this calculation. Also, you can get around the zeros by throwing a substitute in there. Believe it or not this solves the problem..... =substitute(<stuff with zeros as blanks>,"","") Again the absurdity of telling a formula to replace text that exists with the same text is not lost on me. Anyway, filtering the data is not part of the requirement, but certainly something I can see the benefit for and should be optional so good job in displaying!Katha
@JCM wait woa... you used an array on the choosecols formula! NICE~!Katha
My bad, I used the word ‘macro’ mistakenly to refer to the ‘custom formula’, apologies but the message cannot be edited anymore (i need to delete it and rewrite it below):Grenade
[Corrected message]: I finally used this one now in 2024, however as a free advice, because this custom formula returns "0" for empty cells, I used the FILTER function to remove them. Here is what I used in the name manager returning the last column: =LAMBDA(unpivotable; FILTER(unpivotable; CHOOSECOLS(unpivotable; COLUMNS(unpivotable))<>0)Grenade
O
0

I am using this VBA code

Sub Unpivot()
'
Dim Rowlabel As Range
Dim Columnlabel As Range
Dim Pap As Range
Dim Tabl As Range
Dim i As Integer
Dim j As Integer
Dim a As Integer
Dim b As Integer
Dim Data As Range
Dim k As Integer
Dim Label As Range
Dim pvtCache As PivotCache
Dim pvt As PivotTable
Dim SrcData As String
'
ActiveSheet.Copy Before:=Worksheets(1)
Set Tabl = Selection
    For Each Pap In Tabl
     If Pap.MergeCells Then
        With Pap.MergeArea
            .UnMerge
            .Value = Pap.Value
        End With
    End If
    Next
i = Application.InputBox("Number of row contain label:", "Excel", i, Type:=2)
j = Application.InputBox("Number of column contain label:", "Excel", j, Type:=2)
On Error Resume Next
Sheets("Unpivot_Table").Delete
Sheets.Add.Name = "Unpivot_Table"
Set Pap = Range("Unpivot_Table!B2")
b = Tabl.Rows.Count
a = Tabl.Columns.Count
Set Data = Range(Tabl.Cells(i + 1, j + 1), Tabl.Cells(b, a))
Set Columnlabel = Range(Tabl.Cells(i + 1, 1), Tabl.Cells(b, j))
Set Rowlabel = Range(Tabl.Cells(1, j + 1), Tabl.Cells(i, a))
Pap.Select
For Each Column In Data.Columns
    Column.Copy
    Selection.PasteSpecial Paste:=xlPasteValues
    Columnlabel.Copy
    Selection.Offset(0, 1).PasteSpecial Paste:=xlPasteValues
    Column.Copy
    Selection.Offset(b - i, -1).Select
Next Column
Pap.Offset(0, j + 1).Select
For Each Column In Rowlabel.Columns
    Column.Copy
    Range(Selection, Selection.Offset(b - i - 1, 0)).PasteSpecial Paste:=xlPasteValues, Transpose:=True
    Selection.End(xlDown).Offset(1, 0).Select
Next Column
Set Label = Range(Pap.Offset(-1, 0), Pap.Offset(0, i + j + 1))
    For k = 1 To i + j + 1
    Label.Cells(1, k).Value = Application.InputBox(Label.Cells(2, k).Value & " is belong to Fieldname", "Hoang", k, Type:=2)
    Next
Range(Pap.End(xlUp), Pap.End(xlDown).End(xlToRight)).Select
SrcData = ActiveSheet.Name & "!" & Selection.Address
On Error Resume Next
    Sheets("Pivot").Delete
    Sheets.Add.Name = "Pivot"
  Set pvtCache = ActiveWorkbook.PivotCaches.Create( _
    SourceType:=xlDatabase, _
    SourceData:=SrcData)
  Set pvt = pvtCache.CreatePivotTable( _
    TableDestination:="Pivot!" & Sheets("Pivot").Range("A3").Address(ReferenceStyle:=xlR1C1), _
    TableName:="PivotTable1")
End Sub
Overtrade answered 20/11, 2019 at 12:54 Comment(0)
W
0

I believe that you can use a sort of modular arithmetic as follows. Put your data into argument of this UDF with cols and rows legend.

Function MyUnpivot(matice As Range) As Variant
    Dim I As Integer
    Dim J As Integer

    Dim radka As Integer
    Dim sloupec As Integer

    I = matice.Rows.Count - 1
    J = matice.Columns.Count - 1

    Dim returnVal()
    ReDim Preserve returnVal(1 To I * J, 1 To 3)

    For x = 1 To I * J
        radka = ((x - 1) Mod I) + 2
        sloupec = WorksheetFunction.Floor_Math((x - 1 / 2) / I) + 2
        returnVal(x, 1) = matice.Cells(1, sloupec)
        returnVal(x, 2) = matice.Cells(radka, 1)
        returnVal(x, 3) = matice.Cells(radka, sloupec)
    Next

    MyUnpivot = returnVal
End Function
Wallis answered 17/6, 2020 at 9:16 Comment(0)
M
-1

There's another way through Power Query:

  • select your cells
  • menu Data > From a table or a range

screenshot

  • in the Power Query editor, chose all columns save the first one and then Transform > Unpivot

screenshot

  • the table is unpivoted. go to Home > Close and load

screenshot

  • your unpivoted table is here. right-click it and choose Refresh if your original table is updated

screenshot

Mashie answered 13/6, 2019 at 19:35 Comment(0)

© 2022 - 2025 — McMap. All rights reserved.