0

I have jsonb data in Postgres which looks like this:-

{"DestinationLists": [{"name": "TV3/TVNZ/CHOICE", "destinations": [183, 165]}]}

I am trying to write a script/query to change the structure of this json in Postgres to:-

{
  "DestinationLists": [
    {"name": "TV3/TVNZ/CHOICE", "destinations": [{"Id":183}{"Id":165}]}
   ]
}

Is it possible? How do I achieve this?

EDIT: I am trying to update every row of data present in the table with the new structure

This is how the table looks like:-

Id  JsonValue
1   {"DestinationLists": [{"name": "TV3/TVNZ/CHOICE", "destinations": [183, 165]}]}

1 Answer 1

1

You may achieve this with some of the json functions available in postgresql.

In the example with working db-fiddle below I included some additional test data.

Schema (PostgreSQL v13)

CREATE TABLE my_table (
  "dest" json
);

INSERT INTO my_table
  ("dest")
VALUES
  ('{"DestinationLists": [{"name": "TV3/TVNZ/CHOICE", "destinations": [183, 165]}]}'),
  ('{"DestinationLists": [{"name": "SecondTest", "destinations": [103, 105]},{"name": "ThirdTest", "destinations": [3, 5]}]}');

Query #1

WITH expanded_data AS (
    SELECT
        dest::text, 
        json_build_object(
            'name',
             dl->>'name',
             'destinations',
             json_agg(
               json_build_object('Id',dld::text::int)
             )
        ) as dest_list_item
    FROM
        my_table, 
        json_array_elements(dest->'DestinationLists') dl,
        json_array_elements(dl->'destinations') dld
    GROUP BY
        dest::text,dl->>'name'
)
SELECT
    json_build_object(
        'DestinationLists',
        json_agg(dest_list_item)
    ) as new_dest
FROM
    expanded_data
GROUP BY
    dest::text;
new_dest
{"DestinationLists":[{"name":"ThirdTest","destinations":[{"Id":3},{"Id":5}]},{"name":"SecondTest","destinations":[{"Id":103},{"Id":105}]}]}
{"DestinationLists":[{"name":"TV3/TVNZ/CHOICE","destinations":[{"Id":183},{"Id":165}]}]}

View on DB Fiddle

Edit 1

In response to your edit, the code below may be used as an update from statement. NB. The CTE may also be rewritten as a subquery. Please see the example below:

Schema (PostgreSQL v13)

CREATE TABLE my_table (
  id bigserial,
  "dest" jsonb
);

INSERT INTO my_table
  ("dest")
VALUES
  ('{"DestinationLists": [{"name": "TV3/TVNZ/CHOICE", "destinations": [183, 165]}]}'),
  ('{"DestinationLists": [{"name": "SecondTest", "destinations": [103, 105]},{"name": "ThirdTest", "destinations": [3, 5]}]}');

WITH expanded_data AS (
    SELECT
        id, 
        json_build_object(
            'name',
             dl->>'name',
             'destinations',
             json_agg(
               json_build_object('Id',dld::text::int)
             )
        ) as dest_list_item
    FROM
        my_table, 
        jsonb_array_elements(dest->'DestinationLists') dl,
        jsonb_array_elements(dl->'destinations') dld
    GROUP BY
        id,dl->>'name'
), 
new_json AS (
    SELECT
        id,
        json_build_object(
            'DestinationLists',
            json_agg(dest_list_item)
        ) as new_dest
    FROM
        expanded_data
    GROUP BY
        id
)
UPDATE my_table
SET dest = new_json.new_dest
FROM new_json
WHERE my_table.id = new_json.id;

After

SELECT * FROM my_table;
id dest
1 {"DestinationLists":[{"name":"TV3/TVNZ/CHOICE","destinations":[{"Id":183},{"Id":165}]}]}
2 {"DestinationLists":[{"name":"SecondTest","destinations":[{"Id":103},{"Id":105}]},{"name":"ThirdTest","destinations":[{"Id":3},{"Id":5}]}]}

View on DB Fiddle

Edit 2

This edit responds to the edge cases where some destinations may not have destinations and as such may not be updated.

To test, two additional records were added, the sample record provided where there are two named destination lists but only one has destinations and another where there is a named destination list with no destinations.

The update ensures that if there are no changes, that is there are named destination lists with no destinations that these remain the same. It assures this by checking if the number of named destination list items are the same as the number of empty destination list items. Since all are empty, it filters this record from the update and reduces the number of updates required on the database. An example of this is record number 4

The initial query was modified to accommodate these empty lists as they were being filtered with the previous approach.

Schema (PostgreSQL v13)

CREATE TABLE my_table (
  id bigserial,
  "dest" jsonb
);

INSERT INTO my_table
  ("dest")
VALUES
  ('{"DestinationLists": [{"name": "TV3/TVNZ/CHOICE", "destinations": [183, 165]}]}'),
  ('{"DestinationLists": [{"name": "SecondTest", "destinations": [103, 105]},{"name": "ThirdTest", "destinations": [3, 5]}]}'),
  ('{"DestinationLists": [{"name": "TVNZ, Mediaworks, Choice", "destinations": []}, {"name": "TVNZ, Discovery", "destinations": [165, 183, 4155]}]}'),
  ('{"DestinationLists": [{"name": "Fourth Test", "destinations": []}]}');

Query #1

SELECT '------ BEFORE -----';
?column?
------ BEFORE -----

Query #2

SELECT * FROM my_table;
id dest
1 {"DestinationLists":[{"name":"TV3/TVNZ/CHOICE","destinations":[183,165]}]}
2 {"DestinationLists":[{"name":"SecondTest","destinations":[103,105]},{"name":"ThirdTest","destinations":[3,5]}]}
3 {"DestinationLists":[{"name":"TVNZ, Mediaworks, Choice","destinations":[]},{"name":"TVNZ, Discovery","destinations":[165,183,4155]}]}
4 {"DestinationLists":[{"name":"Fourth Test","destinations":[]}]}

Query #3

WITH expanded_data AS (
    SELECT
        id, 
        CASE
            WHEN COUNT(dld)=0 THEN 1
            ELSE 0
        END as name_has_empty_list_item,
        json_build_object(
            'name',
             dl->>'name',
             'destinations',
              CASE 
                  WHEN 
                       COUNT(dld)=0 
                  THEN 
                       to_json(array[]::json[])
                  ELSE
                       json_agg(
                           json_build_object('Id',dld::text::int )
                       )
              END
        ) as dest_list_item
    FROM
        my_table, 
        jsonb_array_elements(dest->'DestinationLists') dl
    LEFT JOIN
        jsonb_array_elements(dl->'destinations') dld ON 1=1
    GROUP BY
        id,dl->>'name'
), 
new_json AS (
    SELECT
        id,
        COUNT(dest_list_item) as no_list_item,
        SUM(name_has_empty_list_item) as no_empty_list_item,
        json_build_object(
            'DestinationLists',
            json_agg(dest_list_item)
        ) as new_dest
    FROM
        expanded_data
    GROUP BY
        id
    HAVING
        SUM(name_has_empty_list_item) <> COUNT(dest_list_item)
    
)
SELECT * FROM new_json;
id no_list_item no_empty_list_item new_dest
1 1 0 {"DestinationLists":[{"name":"TV3/TVNZ/CHOICE","destinations":[{"Id":183},{"Id":165}]}]}
2 2 0 {"DestinationLists":[{"name":"SecondTest","destinations":[{"Id":103},{"Id":105}]},{"name":"ThirdTest","destinations":[{"Id":3},{"Id":5}]}]}
3 2 1 {"DestinationLists":[{"name":"TVNZ, Discovery","destinations":[{"Id":165},{"Id":183},{"Id":4155}]},{"name":"TVNZ, Mediaworks, Choice","destinations":[]}]}

Query #4

WITH expanded_data AS (
    SELECT
        id, 
        CASE
            WHEN COUNT(dld)=0 THEN 1
            ELSE 0
        END as name_has_empty_list_item,
        json_build_object(
            'name',
             dl->>'name',
             'destinations',
              CASE 
                  WHEN 
                       COUNT(dld)=0 
                  THEN 
                       to_json(array[]::json[])
                  ELSE
                       json_agg(
                           json_build_object('Id',dld::text::int )
                       )
              END
        ) as dest_list_item
    FROM
        my_table, 
        jsonb_array_elements(dest->'DestinationLists') dl
    LEFT JOIN
        jsonb_array_elements(dl->'destinations') dld ON 1=1
    GROUP BY
        id,dl->>'name'
), 
new_json AS (
    SELECT
        id,
        json_build_object(
            'DestinationLists',
            json_agg(dest_list_item)
        ) as new_dest
    FROM
        expanded_data
    GROUP BY
        id
    HAVING
        SUM(name_has_empty_list_item) <> COUNT(dest_list_item)
)
UPDATE my_table
SET dest = new_json.new_dest
FROM new_json
WHERE my_table.id = new_json.id;

There are no results to be displayed.


Query #5

SELECT '------ AFTER -----';
?column?
------ AFTER -----

Query #6

SELECT * FROM my_table;
id dest
4 {"DestinationLists":[{"name":"Fourth Test","destinations":[]}]}
1 {"DestinationLists":[{"name":"TV3/TVNZ/CHOICE","destinations":[{"Id":183},{"Id":165}]}]}
2 {"DestinationLists":[{"name":"SecondTest","destinations":[{"Id":103},{"Id":105}]},{"name":"ThirdTest","destinations":[{"Id":3},{"Id":5}]}]}
3 {"DestinationLists":[{"name":"TVNZ, Discovery","destinations":[{"Id":165},{"Id":183},{"Id":4155}]},{"name":"TVNZ, Mediaworks, Choice","destinations":[]}]}

View on DB Fiddle

Let me know if this works for you.

Sign up to request clarification or add additional context in comments.

5 Comments

Hi, many thanks for your answer, this is pretty close to what I am looking for but this its selecting data while I am trying to update the rows
Hi, I have updated the answer and included a working DB fiddle to respond to your updated question. Let me know if this works for you.
The query is ignoring json objects where destinations array is empty, for example: {"DestinationLists": [{"name": "TVNZ, Mediaworks, Choice", "destinations": []}, {"name": "TVNZ, Discovery", "destinations": [165, 183, 4155]}]} Do you think some kind of check can be added to handle this case? Thanks a lot for all the help
@FakharAhmadRasul I've updated the answer with a 2nd Edit using the additional sample record. Let me know if this works for you
I really cannot thank you enough for all the help, I have never worked with Postgresql before and I had to update this data on a production database. Thanks a lot

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.