Flatten a nested JSON with array and filter to CSV

Question

I got a question about How to use Powershell flatten a nested JSON and covert to CSV. Below is my JSON, which is a mail message log get from Office 365 with many users messages, I need to filter the columns, flatten and convert to CSV :

createdDateTime,
receivedDateTime,
from_name
from_adress
To_name_1
To_adress_2
To_name_2
To_adress_2
...

The "from" field has only one data. But the "toRecipients" is a array.

{
  ...
  "createdDateTime": "xxxx-xx-xx",
  "receivedDateTime": "xxxx-xx-xx",
  "isRead": true,
  "from": {
    "emailAddress": {
      "name": "John",
      "adress": "[email protected]"
    }
  },
  "toRecipients": [
    {
      "emailAddress": {
        "name": "Amy",
        "adress": "[email protected]"
      }
    },
    {
      "emailAddress": {
        "name": "Amy",
        "adress": "[email protected]"
      }
    }
  ]
}

Hi, if I understand correctly your filter works and you have trouble by exporting the results as CSV, right? Does this help you (?) : convert-json-to-csv-using-powershell - If you cannot use the solution mentioned in this Post I would try to create a second array and save the filtered json input by creating a new PSObject: $obj = New-Object PSObject — AzureBaum
– AzureBaum, Commented Jun 12, 2020 at 8:22
Have a look at CsvCruncher, although that does not support going through nested arrays. — Ondra Žižka
– Ondra Žižka, Commented Nov 7, 2021 at 4:19

Cbsch · Accepted Answer · 2020-06-12 08:28:00Z

1

Here is a complete runnable example. It will create a file "C:\test.csv".

There is no "automatic" way of flattening a nested object to a flat object. But you can manually create assign properties to a flat object.

First I parse the JSON text into a powershell object

$obj = @"
{
  "createdDateTime": "xxxx-xx-xx",
  "receivedDateTime": "xxxx-xx-xx",
  "isRead": true,
  "from": {
    "emailAddress": {
      "name": "John",
      "adress": "[email protected]"
    }
  },
  "toRecipients": [
    {
      "emailAddress": {
        "name": "Amy",
        "adress": "[email protected]"
      }
    },
    {
      "emailAddress": {
        "name": "Amy",
        "adress": "[email protected]"
      }
    }
  ]
}
"@ | ConvertFrom-Json

Now take the Powershell object (or list of objects, this will work even if you have many of these entries) and pipe it to ForEach-Object. Inside the loop map the different properties to a flat object.

$flattened = $obj | ForEach-Object {
    return [PSCustomObject]@{
        createdDateTime = $_.createdDateTime
        receivedDateTime = $_.receivedDateTime
        from_name = $_.from.emailAddress.name
        from_adress = $_.from.emailAddress.adress
        to_name_1 = $_.toRecipients[0].emailAddress.name
        to_adress_1 = $_.toRecipients[0].emailAddress.adress
        to_name_2 = $_.toRecipients[1].emailAddress.name
        to_adress_2 = $_.toRecipients[1].emailAddress.adress
    }
}

Now you can export the entire thing as a CSV

$flattened | Export-Csv C:\test.csv -Delimiter ";" -Encoding UTF8

This assumes that there will always be 2 toRecipients. It would be possible to dynamically add to_name_3, to_name_4, and so on if more are encountered, but that's quite a bit more complicated.

answered Jun 12, 2020 at 8:28

Cbsch

1,2242 gold badges11 silver badges16 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Eric Chen Over a year ago

Thank you Cbsch for your proper reply, it's really helpful. Yes, that would be a problem and becoming too complicated to get torecipients dynamically. What if I just need to get All nested toRecipients array context to One column in CSV for further analytics?

Cbsch Over a year ago

That would be more easy to do. $_.toRecipients.emailAddress.adress -join ', ' will for example give you all the email addresses as one string value with the addresses separated by a comma.

Eric Chen Over a year ago

Thank you very much Cbsch. I tried your suggested as above, 'To_EmailAddress' = $_.toRecipients.emailAddress.address -join ', ' but this returned weird as below, as others field returned perfectly "System.Object&, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089 Address(int)" For alternative way, I just request to could return all raw data of 'toRecipients' to one CSV column, it also works for me if possible. Tons of thanks for your advice.

Collectives™ on Stack Overflow

Flatten a nested JSON with array and filter to CSV

1 Answer 1

3 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related