Power Query - Loop through unknown number of Tables and append them all - solved

Anonymous
2018-07-17T09:29:18+00:00

Hello,

I have an Excel spreadsheet with a lot of columns but I want to save it in csv format with line breaks to put certain columns on different lines.   This is because I can upload different record types into my database but each record type needs its own line in csv with a header type as the first field on that line.

eg this in Excel:

col1   col2   col3   col4   col5   col6   col7   col8   col9   col10

01        b        c        d       e        02         g       h        03         j

01        r         s         t       u        02         v       w        03         x

becomes this in csv:

01, b, c, d, e

02, g, h

03, j

01, r, s, t, u

02, v, w

03, x              

Is this possible?  Easy?  

thanks

Louise

<The thread has been moved to the correct category by forum moderator>

Microsoft 365 and Office | Excel | For home | Windows

Locked Question. This question was migrated from the Microsoft Support Community. You can vote on whether it's helpful, but you can't add comments or replies or follow the question.

0 comments No comments
Answer accepted by question author
Lz365 38,201 Reputation points Volunteer Moderator
2018-08-01T12:23:56+00:00

This is a very simplified version of this thread. We have an Input Table that looks like:

where each Group of colored columns needs to be transformed in a separate Table (so 3 in this case) and at the end we want the 3 Tables to be Appended (Combined according to Power Query M language)

Challenge

The number of Groups may vary (i.e. from 1 to say 10) from time to time. So we need a query that is flexible enough to handle this variation

Note

In the above example the 1st Group has 3 columns, the 2nd has 4 and the last has 5. To properly Append/Combine Tables they all must have the same number of columns. That aspect of the problem is not detailed in the below Power Query M snipet.

How To

Power Quey M language doesn't offer an easy way to loop. This example uses the List.Generate function to do that the appropriate number of times to build each Table and then Append/Combine them

let

    ...

    GroupsList = Record.ToList(Table.FirstN(Source, 1){0}),

    NbGroups = List.Count(List.Distinct(GroupsList)),

    // Loop (actually NbGroups of times) to create a Table for each "Group"

    // At the end we get a List that contains Table(s)

    The next Step will Append/Combine all Table(s)

    TablesToAppend = List.Generate(

         ()=> [i = -1, j = 0, OutputTable = #table({"Col","Index","Index2"},{})],

            each [i] < NbGroups,

                each

                    [

                        j = [j] +1,

                        ...,              // step1 to build the OutputTable

                        ...,              // step2 to build the OutputTable

                        TableIdx = ...,   // step3 to build the OutputTable

                        OutputTable = Table.AddIndexColumn(TableIdx, "Index2", j, 1),

                        i = [i] +1

                    ],

                each [OutputTable]

                                  ),

    // Append all tables from above List

    TablesAppended = Table.Combine(List.Transform(TablesToAppend, each (_))),

    ...

in

    ...

Was this answer helpful?

2 people found this answer helpful.
0 comments No comments

59 additional answers

Sort by: Newest
  1. Lz365 38,201 Reputation points Volunteer Moderator
    2018-08-01T08:23:10+00:00

    Hi Louise

    Many thanks for all your wishes (going to south west of France close to the Spanish frontier). From a technical perspective I didn't learn anything new until NOW...

    The breaking news of that day is I finally made it work with a looping technic making things fully dynamic (i.e. re. number of "Record Types") meaning there's no more manual adjustment to make anywhere nor anything else. And of course this is way more efficient than what's been provided so far

    I haven't packaged that with the VBA piece of code (this will take a few minutes only)

    yet. Before I share this with you I would appreciate you upload a realistic sample of your Input table with say 5 or 6 "Record Types" so I can test the solution against real data

    Was this answer helpful?

    0 comments No comments
  2. Anonymous
    2018-07-31T16:39:02+00:00

    Hello,

    Fantastic, that worked a treat.  I can easily copy the section to add the new table to Word and find/replace the numbers to increment them.  I've been able to add two more record types/groups and the output is perfect.  I just need to populate with the correct data.

    I really hope this has been a good experience for you in a weird way ;0)

    https://www.dropbox.com/s/4mupxgmkhbg6fat/Thanks.docx?dl=0

    Have a fab holiday, I hope you're going somewhere nice.

    Was this answer helpful?

    0 comments No comments
  3. Lz365 38,201 Reputation points Volunteer Moderator
    2018-07-31T12:10:42+00:00

    Hi again

    Great news, that worked, well done!  Repetitive is fine, it's time to pass the baton to me!  ;0)

    Finally… :-))

    Here is the baton… ;-) In this Zip you will find a NEW version in which I changed just a couple of Step names to make the repetitive work less painfull + fixed a little (probably invisible) issue. There's also a Word document that explains How To do the repetitive work. Good luck...

    If you have any question re. this before Friday feel free to ask

    Was this answer helpful?

    0 comments No comments
  4. Anonymous
    2018-07-31T11:16:08+00:00

    Hello,

    Great news, that worked, well done!  Repetitive is fine, it's time to pass the baton to me!  ;0)

    I understand what you say about the delimiter.  I've used the pipe before to be on the safe side, although we try to ensure there are no commas in this kind of data.

    Was this answer helpful?

    0 comments No comments