Flatten a json datastructure in javascript

Question

I have a json response coming from a service that's a bit of a pain to work with. How can I go about flattening this structure a bit so that "not_restricted":{"doc_count": 976465} is changed to "not_restricted":976465? I'd perfer a function that can detect this kind of a structure in all types of json docs and modify the result.

This is a the general structure of the json:

{
    "took": 159,
    "timed_out": false,
    "_shards": {
        "total": 6,
        "successful": 6,
        "failed": 0
    },
    "hits": {
        "total": 4909332,
        "max_score": 1,
        "hits": [
            {
                ...
            },
            {
                ...
            },
            {
                ...
            }
        ]
    },
    "aggregations": {
        "index_types": {
            "doc_count_error_upper_bound": 0,
            "sum_other_doc_count": 0,
            "buckets": [
                {
                    "key": "client",
                    "doc_count": 1958205,
                    "not_restricted": {
                        "doc_count": 976465
                    },
                    "restricted": {
                        "doc_count": 981740
                    }
                },
                {
                    "key": "ultimateparent",
                    "doc_count": 1616164,
                    "not_restricted": {
                        "doc_count": 741059
                    },
                    "restricted": {
                        "doc_count": 875105
                    }
                },
                {
                    "key": "facility",
                    "doc_count": 1334963,
                    "not_restricted": {
                        "doc_count": 914090
                    },
                    "restricted": {
                        "doc_count": 420872
                    }
                }
            ]
        }
    }
}

@HorseVoice you might want to checkout lodash.com/docs#flatten and lodash.com/docs#flattenDeep — Josh C.
– Josh C., Commented Oct 19, 2015 at 21:10

Bergi · Accepted Answer · 2015-10-20 15:41:24Z

5

You can very easily do this by using the reviver parameter of JSON.parse, which allows you to filter every object in the JSON:

var res = JSON.parse(json, function(k, o) {
    if (Object(o) !== o) return o; // primitive values
    var keys = Object.keys(o);
    if (keys.length == 1 && keys[0] == "doc_count")
        return o.doc_count;
    else
        return o;
});

You can also test k to include "restricted" or so if you want to make sure that doc_count singletons are only replaced in such properties.

edited Oct 20, 2015 at 15:41

answered Oct 19, 2015 at 21:15

Bergi

671k162 gold badges1k silver badges1.5k bronze badges

Sign up to request clarification or add additional context in comments.

9 Comments

Horse Voice Over a year ago

Bergi -I'm not sure what this does as I'm new to javascript, but does this modify the existing doc? Lets say json argument is the response doc. Does it just modify it? or is this just returning the aggs portion of it?

Bergi Over a year ago

No, it returns a parsed version (object) of the json string in which each {"doc_count":…} object is replaced by the … only. Btw, you cannot "modify" strings in JS, they're immutable. If you need to get JSON back, you can use JSON.stringify(res, null, 4); and if you want you can overwrite the json variable with it.

Horse Voice Over a year ago

So it returns the entire doc? just with the slight modification of doc_count?

Bergi Over a year ago

Yes, it returns an object for the complete JSON without the doc_count objects. Just try it out and inspect the result in your console :-)

Horse Voice Over a year ago

How would I make this as a predefined method? I'm using extjs framework and it is not understanding this functional way of doing this. How can I make it so that I define a specific method that does just this, and then invoke it where I need it in the code, by passing in the response and this method returns the response? Also I don't think JSON.parse is working in my IE 8 old enterprise .

|

Eric Hartford · Accepted Answer · 2015-10-20 16:58:44Z

1

You can use LoDash to make your life easier. Include it, and then you can use _.each

_.each(data.aggregations.index_types.buckets, function(n){
  n.not_restricted = n.not_restricted.doc_count;
  n.restricted = n.restricted.doc_count;
})

edited Oct 20, 2015 at 16:58

answered Oct 19, 2015 at 21:20

Eric Hartford

18.2k4 gold badges35 silver badges51 bronze badges

Comments

tymeJV · Accepted Answer · 2015-10-19 21:07:40Z

0

If you want to go thru the object and flatten all 1 property objects, you can do:

for (var i = 0; i < data.aggregations["index_types"].buckets.length; i++) {
    var obj = data.aggregations["index_types"].buckets[i];
    for (var key in obj) {
        if (typeof obj[key] === "object" && Object.keys(obj[key]).length === 1) {
            //1 property object!
            obj[key] = obj[key][Object.keys(obj[key])[0]];
        } 
    }
}

Demo: http://jsfiddle.net/jkm1vmbm/

answered Oct 19, 2015 at 21:07

tymeJV

105k14 gold badges165 silver badges158 bronze badges

5 Comments

Horse Voice Over a year ago

It's actually a json string. Not really an object. Would It work the same way? sorry I'm new to javascript

tymeJV Over a year ago

@HorseVoice -- Use JSON.parse(str) to convert it to an object - it won't work the same if it's just a string.

Horse Voice Over a year ago

then I'd have to convert that back to a string for processing. for that I can just do String(jsonDoc) right?

tymeJV Over a year ago

@HorseVoice -- JSON.stringify(obj)

Horse Voice Over a year ago

This only returns the modified object and forgets the rest of the json. This does not work for me. I need the entire doc back with the flattened doc_count garbage inside restricted and unrestricted

nem035 · Accepted Answer · 2016-10-15 20:20:23Z

0

Since you said that the JSON are actually strings, if all you need is that specific replacement:

Replacing `{"doc_count": NUMBER}` with that `NUMBER`

What you can do is just use a regex to replace what you want:

s.replace(/({"doc_count":([0-9]+)})/g, '$2')

Which works like:

(                   // start capturing group 1
{\"doc_count\":     // match the string {\"doc_count\":
    (               // start capturing group 2
    [0-9]+          // match one or more digits
    )               // close capturing group 2
}                   // add the rest of the string (a closing bracket)

Then you can use $2 in your replace() to extract only the number from the capturing group 2.

var json = {
  "aggregations": {
    "index_types": {
      "doc_count_error_upper_bound": 0,
      "sum_other_doc_count": 0,
      "buckets": [{
        "key": "client",
        "doc_count": 1958205,
        "not_restricted": {
          "doc_count": 976465
        },
        "restricted": {
          "doc_count": 981740
        }
      }, {
        "key": "ultimateparent",
        "doc_count": 1616164,
        "not_restricted": {
          "doc_count": 741059
        },
        "restricted": {
          "doc_count": 875105
        }
      }, {
        "key": "facility",
        "doc_count": 1334963,
        "not_restricted": {
          "doc_count": 914090
        },
        "restricted": {
          "doc_count": 420872
        }
      }]
    }
  }
};

var s = JSON.stringify(json);
// replace all instances of {doc_count: NUMBER} with that NUMBER
console.log(s.replace(/({"doc_count":([0-9]+)})/g, '$2'));

I also noticed that you have properties doc_count outside of restricted and non_restricted. If you only want to replace the ones within a property restricted or non_restricted, You can use this regex:

("((?:not_)?restricted"):{"doc_count":([0-9]+)})

That added part at the beginning basically means "starts with 'restricted' with an optional 'not_' before it"

It's used like:

s.replace(/("((?:not_)?restricted"):{"doc_count":([0-9]+)})/g, '"$2:$3')

var json = {
  "aggregations": {
    "index_types": {
      "doc_count_error_upper_bound": 0,
      "sum_other_doc_count": 0,
      "buckets": [{
        "key": "client",
        "doc_count": 1958205,
        "not_restricted": {
          "doc_count": 976465
        },
        "restricted": {
          "doc_count": 981740
        }
      }, {
        "key": "ultimateparent",
        "doc_count": 1616164,
        "not_restricted": {
          "doc_count": 741059
        },
        "restricted": {
          "doc_count": 875105
        }
      }, {
        "key": "facility",
        "doc_count": 1334963,
        "not_restricted": {
          "doc_count": 914090
        },
        "restricted": {
          "doc_count": 420872
        }
      }]
    }
  }
};

var s = JSON.stringify(json);
console.log(s.replace(/("((?:not_)?restricted"):{"doc_count":([0-9]+)})/g, '"$2:$3'));

NOTE: using ?: for the (?:not_) makes the part in parenthesis a non-capturing group so it doesn't get counted in the capturing groups. The term "restricted" with the optional "not_" before it is now contained in group 2 and the number value is in group 3. We also need to prepend a " and combine the two groups with a :.

edited Oct 15, 2016 at 20:20

answered Oct 19, 2015 at 21:14

nem035

35.5k6 gold badges93 silver badges104 bronze badges

9 Comments

nem035 Over a year ago

It is a regex capturing group. Essentially, you can capture portions (or groups) of the matches.

nem035 Over a year ago

@HorseVoice I added much more explanation, hopefully that helps

Horse Voice Over a year ago

Yea, all the other structure MUST be same. Only restricted and unrestricted. There are other fields on the doc and lists etc, with nested doc_counts that I DONT want to mess with at all. This is very specific. How do I use this? ("(?:not_)?restricted":{"doc_count":([0-9]+)})

nem035 Over a year ago

The same way you would use the other regex. I edited my answer accordingly.

Horse Voice Over a year ago

Okay, this did not do anything. The response was unmodified. And I still see doc_count inside of restricted and not_restricted

|

Collectives™ on Stack Overflow

Flatten a json datastructure in javascript

4 Answers 4

9 Comments

Comments

5 Comments

9 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

9 Comments

Comments

5 Comments

9 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related