Converting a CSV file into a nested JSON file using ./jq

Question

I am hoping I can get an answer on this question. I am using ./jq to convert a CSV file into a file of JSON objects. Each line of the CSV input represents a JSON object. A sample of the CSV input is as follows:

loyalty_card_num,extended_data_source,is_pharmacy_linked,first_linked_date,linked_store,is_caregiver,caregiver_first_linked_date,caregiver_store
6105700000000170419,PharmacyLinks,false,,,false,,
6105700000006125318,PharmacyLinks,True,2022-12-27,7236,false,,

The resulting JSON file I am trying to create is:

{"loyaltyID":"6105700000000170419","extendedDataSource":"PharmacyLinks","extendedData":{"firstLinkDate":"","isCaregiver":"false","caregiverStores":"","caregiverFirstLinkDate":"","isPharmacyLinked":"false","linkedStores":""}}
{"loyaltyID":"6105700000006125318","extendedDataSource":"PharmacyLinks","extendedData":{"firstLinkDate":"2022-12-27T00:00:00-0500","isCaregiver":"false","caregiverStores":"","caregiverFirstLinkDate":"","isPharmacyLinked":"True","linkedStores":"7236"}}

My ./jq code is:

jq --slurp --raw-input --raw-output \
  'split("\n") | .[1:] | map(split(",")) |
    map(
        {
            "loyaltyID": .[0],
            "extendedDataSource": .[1],
            "extendedData":
            {
                "firstLinkDate": .[3],
                "isCaregiver": .[5],
                "caregiverStores": .[7],
                "caregiverFirstLinkDate": .[6],
                "isPharmacyLinked": .[2],
                "linkedStores": .[4]
            }
        }
    )' \
  short.csv > short.json

The result of the above code creates an array of nested JSON object:

[
  {
    "loyaltyID": "6105700000000170419",
    "extendedDataSource": "PharmacyLinks",
    "extendedData": {
      "firstLinkDate": "",
      "isCaregiver": "false",
      "caregiverStores": "",
      "caregiverFirstLinkDate": "",
      "isPharmacyLinked": "false",
      "linkedStores": ""
    }
  },
  {
    "loyaltyID": "6105700000006125318",
    "extendedDataSource": "PharmacyLinks",
    "extendedData": {
      "firstLinkDate": "2022-12-27",
      "isCaregiver": "false",
      "caregiverStores": "",
      "caregiverFirstLinkDate": "",
      "isPharmacyLinked": "True",
      "linkedStores": "7236"
    }
  }
]

I don't want an array but a separate JSON object for each elements in the array. Using the -c switch will just put the entire output on one line. So my goal is to get rid of the [] and have one object per line. Thank you in advance.

pmf · Accepted Answer · 2025-10-17 05:34:34Z

5

Just don't map and you get the items directly. Also, instead of --slurping the entire input at once, then splitting again, you could just walk line by line. Using inputs without the -n flag ignores the first line:

< input.csv jq -Rc 'inputs / "," |  {
  "loyaltyID": .[0],
  "extendedDataSource": .[1],
  "extendedData": {
    "firstLinkDate": .[3],
    "isCaregiver": .[5],
    "caregiverStores": .[7],
    "caregiverFirstLinkDate": .[6],
    "isPharmacyLinked": .[2],
    "linkedStores": .[4]
  }
}'

{"loyaltyID":"6105700000000170419","extendedDataSource":"PharmacyLinks","extendedData":{"firstLinkDate":"","isCaregiver":"false","caregiverStores":"","caregiverFirstLinkDate":"","isPharmacyLinked":"false","linkedStores":""}}
{"loyaltyID":"6105700000006125318","extendedDataSource":"PharmacyLinks","extendedData":{"firstLinkDate":"2022-12-27","isCaregiver":"false","caregiverStores":"","caregiverFirstLinkDate":"","isPharmacyLinked":"True","linkedStores":"7236"}}

Demo

answered Oct 17 at 5:34

pmf

38.3k3 gold badges31 silver badges41 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

sunpack Oct 17 at 6:03

Thanks PMF for the quick reply. That worked and less code.

Rehan Hanif · Accepted Answer · 2025-10-17 05:37:07Z

4

Do this: emit each object instead of the array

Add .[] to the end of your filter so jq outputs each element separately.

jq --slurp --raw-input --raw-output '
  split("\n") | .[1:] | map(split(",")) |
  map({
    "loyaltyID": .[0],
    "extendedDataSource": .[1],
    "extendedData": {
      "firstLinkDate": .[3],
      "isCaregiver": .[5],
      "caregiverStores": .[7],
      "caregiverFirstLinkDate": .[6],
      "isPharmacyLinked": .[2],
      "linkedStores": .[4]
    }
  })[]
' short.csv > short.json

This prints one JSON object after another (no surrounding []).

If you prefer JSON Lines (one compact object per line), add -c:

jq -c --slurp --raw-input '
  split("\n") | .[1:] | map(split(",")) |
  map({ "loyaltyID": .[0], "extendedDataSource": .[1],
        "extendedData": { "firstLinkDate": .[3], "isCaregiver": .[5],
          "caregiverStores": .[7], "caregiverFirstLinkDate": .[6],
          "isPharmacyLinked": .[2], "linkedStores": .[4] } })[]
' short.csv > short.jsonl

You can try this: if your CSV ends with a blank line, use .[1:-1] instead of .[1:] to drop it.

Or Just do ti:

Add [] after the map(...) to stream objects.
Use -c only if you want one-line-per-object output.

answered Oct 17 at 5:37

Rehan Hanif

413 bronze badges

1 Comment

sunpack Oct 17 at 5:56

That was perfect solved my problem. I used the -c switch since I need for it to be one per line. Thank you

Soroosh Mirzasarvari · Accepted Answer · 2025-11-08 08:18:51Z

0

Maybe, this is a better soluion:

tail +2 short.csv | jq --raw-input \
  'split(",") | { 
    "loyaltyID": .[0],
    "extendedDataSource": .[1],
    "extendedData":{
      "firstLinkDate": .[3],
      "isCaregiver": .[5],
      "caregiverStores": .[7],
      "caregiverFirstLinkDate": .[6],
      "isPharmacyLinked": .[2],
      "linkedStores": .[4]
    }
  }' > short.json

Overall, my experience shows that if you want to parse a file line-by-line, the better solution is to use other tools like cat and tail.
In the above solution, jq should only care about each JSON object individually so it's less code in the query. Hope this is useful.

edited Nov 8 at 8:18

answered Nov 5 at 17:04

Soroosh Mirzasarvari

14 bronze badges

1 Comment

Jeremy Caney Nov 5 at 19:13

Thank you for contributing to the Stack Overflow community. This may be a correct answer, but it’d be really useful to provide additional explanation of your code so developers can understand your reasoning. (E.g., why do you think it’s a better solution?) This is especially useful for new developers who aren’t as familiar with the syntax or struggling to understand the concepts. Would you kindly edit your answer to include additional details for the benefit of the community?

Collectives™ on Stack Overflow

Converting a CSV file into a nested JSON file using ./jq

3 Answers 3

1 Comment

1 Comment

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

1 Comment

1 Comment

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related