Parse JSON HTTP response using golang

Question

I am trying to get the value of say "ip" from my following curl output:

{  
  "type":"example",
  "data":{  
    "name":"abc",
    "labels":{  
      "key":"value"
    }
  },
  "subsets":[  
    {  
      "addresses":[  
        {  
          "ip":"192.168.103.178"
        }
      ],
      "ports":[  
        {  
          "port":80
        }
      ]
    }
  ]
}

I have found many examples in the internet to parse json output of curl requests and I have written the following code, but that doesn't seem to return me the value of say "ip"

package main

import (
"encoding/json"
"fmt"
"io/ioutil"
"log"
"net/http"
"time"
)

type svc struct {
    Ip string `json:"ip"`
}

func main() {

url := "http://myurl.com"

testClient := http.Client{
    Timeout: time.Second * 2, // Maximum of 2 secs
}

req, err := http.NewRequest(http.MethodGet, url, nil)
if err != nil {
    log.Fatal(err)
}


res, getErr := testClient.Do(req)
if getErr != nil {
    log.Fatal(getErr)
}

body, readErr := ioutil.ReadAll(res.Body)
if readErr != nil {
    log.Fatal(readErr)
}

svc1 := svc{}
jsonErr := json.Unmarshal(body, &svc1)
if jsonErr != nil {
    log.Fatal(jsonErr)
}

fmt.Println(svc1.Ip)
}

I would appreciate if anyone could provide me hints on what I need to add to my code to get the value of say "ip".

Your json is not valid., you have one extra } right below ip, and also commas (,) on the last item of two objects (after 80 and after "192.168.103.178"). You could use any online json formatter/validator to help you build a valid json - just google it and you will find several options. — Leonardo Alves Machado
– Leonardo Alves Machado, Commented Aug 18, 2017 at 11:57
Sorry the extra bracket below ip and the extra commas were just a copy & paste fault; the actual curl output is very large so had to shorten it for the sake of this question. I have fixed it in the question now. — rmb
– rmb, Commented Aug 18, 2017 at 12:00
Your struct does not match the structure of the JSON input. You're trying to deserialize a large data structure where the ip field is deeply nested inside arrays into a single object with a single top-level field that holds a single value - what if there are multiple items in subsets or addresses? How is it supposed to know that subsets[n].addresses[m].ip is supposed to map to .ip? — Adrian
– Adrian, Commented Aug 18, 2017 at 13:30
It also seems like you're confusing the curl Linux utility with Go's http.Client. They are not at all the same thing. — Adrian
– Adrian, Commented Aug 18, 2017 at 13:31
@Adrian Yes you are right, there are multiple items under addresses in the actual curl output but what I am interested in is subsets[n].addresses[m].ip that will always map to an IP. — rmb
– rmb, Commented Aug 18, 2017 at 13:37

Mike · Accepted Answer · 2020-11-29 20:41:45Z

17

You can create structs which reflect your json structure and then decode your json.

package main

import (
    "bytes"
    "encoding/json"
    "fmt"
    "log"
)

type Example struct {
    Type    string   `json:"type,omitempty"`
    Subsets []Subset `json:"subsets,omitempty"`
}

type Subset struct {
    Addresses []Address `json:"addresses,omitempty"`
}

type Address struct {
    IP string `json:"IP,omitempty"`
}

    func main() {

    m := []byte(`{"type":"example","data": {"name": "abc","labels": {"key": "value"}},"subsets": [{"addresses": [{"ip": "192.168.103.178"}],"ports": [{"port": 80}]}]}`)

    r := bytes.NewReader(m)
    decoder := json.NewDecoder(r)

    val := &Example{}
    err := decoder.Decode(val)

    if err != nil {
        log.Fatal(err)
    }

    // If you want to read a response body
    // decoder := json.NewDecoder(res.Body)
    // err := decoder.Decode(val)

    // Subsets is a slice so you must loop over it
    for _, s := range val.Subsets {
        // within Subsets, address is also a slice
        // then you can access each IP from type Address
        for _, a := range s.Addresses {
            fmt.Println(a.IP)
        }
    }

}

The output would be: 192.168.103.178

By decoding this to a struct, you can loop over any slice and not limit yourself to one IP

Example here:

https://play.golang.org/p/sWA9qBWljA

edited Nov 29, 2020 at 20:41

answered Aug 18, 2017 at 13:35

Mike

2,0643 gold badges26 silver badges43 bronze badges

Sign up to request clarification or add additional context in comments.

8 Comments

rmb Over a year ago

Thanks for such structured method! However in my case I have json output from a url instead of m that you have defined(as shown in my question). In that case, how do I modify the 25th line of your code?

Mike Over a year ago

I'll update my answer to include an example where I read in a response body

rmb Over a year ago

A question- how can one retrieve the ip (i.e 192.168.103.178) from [{[{192.168.103.178}]}]?I tried the strings.Trim function but that complains cannot use ip (type []Subset) as type string in argument to strings.Trim

Mike Over a year ago

I've updated the example in the go playground, check that link out

Mike Over a year ago

The new code @ play.golang.org/p/sWA9qBWljA returns exactly 192.168.103.178, check it out ;)

|

putu · Accepted Answer · 2017-08-18 13:03:59Z

One approach is to unmarshal the JSON to a map, e.g. (assumes jsData contains JSON string)

obj := map[string]interface{}{}
if err := json.Unmarshal([]byte(jsData), &obj); err != nil {
    log.Fatal(err)
}

Next, implement a function for searching the value associated with a key from the map recursively, e.g.

func find(obj interface{}, key string) (interface{}, bool) {
    //if the argument is not a map, ignore it
    mobj, ok := obj.(map[string]interface{})
    if !ok {
        return nil, false
    }

    for k, v := range mobj {
        //key match, return value
        if k == key {
            return v, true
        }

        //if the value is a map, search recursively
        if m, ok := v.(map[string]interface{}); ok {
            if res, ok := find(m, key); ok {
                return res, true
            }
        }
        //if the value is an array, search recursively 
        //from each element
        if va, ok := v.([]interface{}); ok {
            for _, a := range va {
                if res, ok := find(a, key); ok {
                    return res,true
                }
            }
        }
    }

    //element not found
    return nil,false
}

Note, that the above function return an interface{}. You need to convert it to appropriate type, e.g. using type switch:

if ip, ok := find(obj, "ip"); ok {
    switch v := ip.(type) {
    case string:
        fmt.Printf("IP is a string -> %s\n", v)
    case fmt.Stringer:
        fmt.Printf("IP implements stringer interface -> %s\n", v.String())
    case int:

    default:
        fmt.Printf("IP = %v, ok = %v\n", ip, ok)
    }
}

A working example can be found at https://play.golang.org/p/O5NUi4J0iR

Nimantha · Accepted Answer · 2021-11-02 09:34:42Z

Typically in these situations you will see people describe all of these sub struct types. If you don't actually need to reuse the definition of any sub structs (like as a type for a function argument), then you don't need to define them. You can just use one definition for the whole response. In addition, in some cases you don't need to define a type at all, you can just do it at the time of declaration:

package main
import "encoding/json"

const s = `
{  
   "subsets": [
      {  
         "addresses": [ 
            {"ip": "192.168.103.178"}
         ]
      }
   ]
}
`

func main() {
   var svc struct {
      Subsets []struct {
         Addresses []struct { Ip string }
      }
   }
   json.Unmarshal([]byte(s), &svc)
   ip := svc.Subsets[0].Addresses[0].Ip
   println(ip == "192.168.103.178")
}

bayrinat · Accepted Answer · 2017-08-18 13:07:20Z

2

You can write your own decoder or use existing third-party decoders. For instance, github.com/buger/jsonparser could solve your problem by iterating throw array (two times).

package main

import (
    "github.com/buger/jsonparser"
    "fmt"
)

var data =[]byte(`{
  "type":"example",
  "data":{
    "name":"abc",
    "labels":{
      "key":"value"
    }
  },
  "subsets":[
    {
      "addresses":[
        {
          "ip":"192.168.103.178"
        }
      ],
      "ports":[
        {
          "port":80
        }
      ]
    }
  ]
}`)

func main() {
    jsonparser.ArrayEach(data, func(value []byte, dataType jsonparser.ValueType, offset int, err error) {
        jsonparser.ArrayEach(value, func(value []byte, dataType jsonparser.ValueType, offset int, err error) {
            v, _, _, err := jsonparser.Get(value, "ip")
            if err != nil {
                return
            }
            fmt.Println("ip: ", string(v[:]))
        }, "addresses")
    }, "subsets")
}

Output: ip: 192.168.103.178

answered Aug 18, 2017 at 13:07

bayrinat

1,6289 silver badges18 bronze badges

2 Comments

Adrian Over a year ago

This seems like an unnecessarily heavy-weight solution to a simple data structure mismatch.

bayrinat Over a year ago

@anuribs data, _ := ioutil.ReadAll(res.Body)

Collectives™ on Stack Overflow

Parse JSON HTTP response using golang

4 Answers 4

8 Comments

Comments

Comments

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

8 Comments

Comments

Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related