Power Query - Loop through unknown number of Tables and append them all - solved

Anonymous
2018-07-17T09:29:18+00:00

Hello,

I have an Excel spreadsheet with a lot of columns but I want to save it in csv format with line breaks to put certain columns on different lines.   This is because I can upload different record types into my database but each record type needs its own line in csv with a header type as the first field on that line.

eg this in Excel:

col1   col2   col3   col4   col5   col6   col7   col8   col9   col10

01        b        c        d       e        02         g       h        03         j

01        r         s         t       u        02         v       w        03         x

becomes this in csv:

01, b, c, d, e

02, g, h

03, j

01, r, s, t, u

02, v, w

03, x              

Is this possible?  Easy?  

thanks

Louise

<The thread has been moved to the correct category by forum moderator>

Microsoft 365 and Office | Excel | For home | Windows

Locked Question. This question was migrated from the Microsoft Support Community. You can vote on whether it's helpful, but you can't add comments or replies or follow the question.

0 comments No comments
Answer accepted by question author
Lz365 38,201 Reputation points Volunteer Moderator
2018-08-01T12:23:56+00:00

This is a very simplified version of this thread. We have an Input Table that looks like:

where each Group of colored columns needs to be transformed in a separate Table (so 3 in this case) and at the end we want the 3 Tables to be Appended (Combined according to Power Query M language)

Challenge

The number of Groups may vary (i.e. from 1 to say 10) from time to time. So we need a query that is flexible enough to handle this variation

Note

In the above example the 1st Group has 3 columns, the 2nd has 4 and the last has 5. To properly Append/Combine Tables they all must have the same number of columns. That aspect of the problem is not detailed in the below Power Query M snipet.

How To

Power Quey M language doesn't offer an easy way to loop. This example uses the List.Generate function to do that the appropriate number of times to build each Table and then Append/Combine them

let

    ...

    GroupsList = Record.ToList(Table.FirstN(Source, 1){0}),

    NbGroups = List.Count(List.Distinct(GroupsList)),

    // Loop (actually NbGroups of times) to create a Table for each "Group"

    // At the end we get a List that contains Table(s)

    The next Step will Append/Combine all Table(s)

    TablesToAppend = List.Generate(

         ()=> [i = -1, j = 0, OutputTable = #table({"Col","Index","Index2"},{})],

            each [i] < NbGroups,

                each

                    [

                        j = [j] +1,

                        ...,              // step1 to build the OutputTable

                        ...,              // step2 to build the OutputTable

                        TableIdx = ...,   // step3 to build the OutputTable

                        OutputTable = Table.AddIndexColumn(TableIdx, "Index2", j, 1),

                        i = [i] +1

                    ],

                each [OutputTable]

                                  ),

    // Append all tables from above List

    TablesAppended = Table.Combine(List.Transform(TablesToAppend, each (_))),

    ...

in

    ...

Was this answer helpful?

2 people found this answer helpful.
0 comments No comments

59 additional answers

Sort by: Most helpful
  1. Lz365 38,201 Reputation points Volunteer Moderator
    2018-08-01T14:16:31+00:00

    Thanks for changing the title and marking the right post. All the best again...

    Was this answer helpful?

    0 comments No comments
  2. Anonymous
    2018-08-01T12:59:02+00:00

    Thanks for the pointer.  I hope you get some easy queries for a bit!

    Was this answer helpful?

    0 comments No comments
  3. Lz365 38,201 Reputation points Volunteer Moderator
    2018-08-01T12:50:52+00:00

    Actually Power Query can do a lot of complex things especially in term of transformations and data cleansing… Experts on that topic are "reachable" on the Technet/PowerQuery forum

    The post I was talking about earlier (the summary) is already there, just before your previous answer.

    Wish you all the best with your next challenge...

    Was this answer helpful?

    0 comments No comments