1

Problem: Create json data structure from values stored from a rest api GET

Issue: Creating the nested data structure dynamically

Python Code for retrieving values:

url = 'https://ciscoprime.com/webacs/api/v2/data/ClientDetails.json?.nocount=True&.maxResults=2&.firstResult=0&.full=true'
get_response = requests.get(url, verify=False)
response = json.dumps(get_response.json())
resp = json.loads(response)

#variable = resp['queryResponse']['entity'][i]['clientDetailsDTO']['']

i = 0
clients = {}
while i < len(resp['queryResponse']['entity']):
    clientID = resp.get("queryResponse", {}).get("entity", {})[i].get("clientDetailsDTO", {}).get("@id")
    associationTime = resp.get("queryResponse", {}).get("entity", {})[i].get("clientDetailsDTO", {}).get("associationTime")
    clientInterface = resp.get("queryResponse", {}).get("entity", {})[i].get("clientDetailsDTO", {}).get("clientInterface")
    connectionType = resp.get("queryResponse", {}).get("entity", {})[i].get("clientDetailsDTO", {}).get("connectionType")
    deviceIpAddress = resp.get("queryResponse", {}).get("entity", {})[i].get("clientDetailsDTO", {}).get("deviceIpAddress")
    deviceName = resp.get("queryResponse", {}).get("entity", {})[i].get("clientDetailsDTO", {}).get("deviceName")
    firstSeenTime = resp.get("queryResponse", {}).get("entity", {})[i].get("clientDetailsDTO", {}).get("firstSeenTime")
    ifIndex = resp.get("queryResponse", {}).get("entity", {})[i].get("clientDetailsDTO", {}).get("ifIndex")
    ipAddress = resp.get("queryResponse", {}).get("entity", {})[i].get("clientDetailsDTO", {}).get("ipAddress")
    macAddress = resp.get("queryResponse", {}).get("entity", {})[i].get("clientDetailsDTO", {}).get("macAddress")
    updateTime = resp.get("queryResponse", {}).get("entity", {})[i].get("clientDetailsDTO", {}).get("updateTime")
    vendor = resp.get("queryResponse", {}).get("entity", {})[i].get("clientDetailsDTO", {}).get("vendor")
    vlan = resp.get("queryResponse", {}).get("entity", {})[i].get("clientDetailsDTO", {}).get("vlan")
    vlanName = resp.get("queryResponse", {}).get("entity", {})[i].get("clientDetailsDTO", {}).get("vlanName")
    #print 'ClientID: {0}, associationTime: {1}, vlanName: {2}'.format(clientID, associationTime, vlanName)

    i += 1

GET format is in json

{"queryResponse": {"@responseType": "listEntityInstances", 
"@count": 192077, "@last": 1, "@rootUrl": "https://ciscopri
me.com/webacs/api/v2/data", "@requestUrl": "https://c
iscoprime.com/webacs/api/v2/data/ClientDetails?.nocou
nt=True&amp;.maxResults=2&amp;.firstResult=0&amp;.full=true
", "entity": [{"clientDetailsDTO": {"apName": "nzcfd-rotoof
f-ap2", "deviceName": "cntl-1", "updateTime":
 1490290426910, "protocol": "DOT11N2_4GNZ", "clientInterfac
e": "management", "macAddress": "FF:FF:FF:FF:FF:FF", "ccxLS
Version": "V1", "policyTypeStatus": "NOMETHOD", "wgbMacAddr
ess": "00:00:00:00:00:00", "wepState": "DISABLED", "ccxVers
ion": "UNSUPPORTED", "speed": "UNKNOWN", "@displayName": "4
110106", "hreapLocallyAuthenticated": "2", "ssid": "Gue
st", "clientAaaOverrideAclApplied": "NA", "ccxVSVersion": "
V1", "securityPolicyStatus": "FAILED", "authenticationAlgor
ithm": "OPENSYSTEM", "connectionType": "LIGHTWEIGHTWIRELESS
", "postureStatus": "UNKNOWN", "deviceType": "none", "locat
ion": "Root Area", "wiredClientType": "NA", "status": "ASSO
CIATED", "firstSeenTime": 1487771895967, "wgbStatus": "REGU
LARCLIENT", "vendor": "Samsung Electronics", "vlan": "10", 
"ccxMSVersion": "V1", "nacState": "ACCESS", "webSecurity": 
"DISABLED", "policyType": "NA", "associationTime": 14881782
85705, "ifIndex": 0, "apIpAddress": {"address": "1.2.2.2"}, "@id": 4110106, "eapType": "UNNOWN", "mobilityStatus"
: "EXPORTANCHORED", "ipType": "IPV4", "ipAddress": {"addres
s": "10.2.2.2"}, "apSlotId": 0, "encryptionCypher": "
NONE", "apMacAddress": "FF:FF:FF:FF:FF:FF", "ccxFSVersion":
 "V1", "clientApMode": "UNKNOWN", "deviceIpAddress": {"addr
ess": "172.2.2.2"}, "clientAclApplied": "NA"}, "@url": "
https://ciscoprime.com/webacs/api/v2/data/ClientDetai
ls/4110106", "@type": "ClientDetails", "@dtoType": "clientD
etailsDTO"}

Desired Data Structure ouput.

clientID : value
{
    associationTime : value
    clientInterface : value
    etc...
}
clientID : value
{
    associationTime : value
    clientInterface : value
    etc...    
}
etc....
Improve this question
5
  • You need to provide the format of the json return data from the API GET request. Commented Mar 23, 2017 at 17:28
  • I have updated the question with the requested information Commented Mar 23, 2017 at 17:40
  • 1
    Your desired data structure output is not JSON. Commented Mar 23, 2017 at 17:43
  • I have removed that requirement as it does not pertain to this. @SatishGarg Commented Mar 23, 2017 at 17:46
  • The json you have: clientID: value { stuff} isn't valid. So you can't do that. But you can have the actual client id be the key to a dictionary object. Commented Mar 23, 2017 at 18:24

2 Answers 2

Reset to default
1

The code below does what you want. It finds the id field, then iterates through all the keys and adds them to a dict, adding id:{values} to the master dict, which we print at the end.

import json

my_response = {"queryResponse": {"@responseType": "listEntityInstances", "@count": 192077, "@last": 1,
                                 "@rootUrl": "https://ciscoprime.com/webacs/api/v2/data",
                                 "@requestUrl": "https://ciscoprime.com/webacs/api/v2/data/ClientDetails?.nocount=True&amp;.maxResults=2&amp;.firstResult=0&amp;.full=true",
                                 "entity": [{"clientDetailsDTO": {"apName": "nzcfd-rotooff-ap2", "deviceName": "cntl-1",
                                                                  "updateTime": 1490290426910,
                                                                  "protocol": "DOT11N2_4GNZ",
                                                                  "clientInterface": "management",
                                                                  "macAddress": "FF:FF:FF:FF:FF:FF",
                                                                  "ccxLSVersion": "V1", "policyTypeStatus": "NOMETHOD",
                                                                  "wgbMacAddress": "00:00:00:00:00:00",
                                                                  "wepState": "DISABLED", "ccxVersion": "UNSUPPORTED",
                                                                  "speed": "UNKNOWN", "@displayName": "4110106",
                                                                  "hreapLocallyAuthenticated": "2", "ssid": "Guest",
                                                                  "clientAaaOverrideAclApplied": "NA",
                                                                  "ccxVSVersion": "V1",
                                                                  "securityPolicyStatus": "FAILED",
                                                                  "authenticationAlgorithm": "OPENSYSTEM",
                                                                  "connectionType": "LIGHTWEIGHTWIRELESS",
                                                                  "postureStatus": "UNKNOWN", "deviceType": "none",
                                                                  "location": "Root Area", "wiredClientType": "NA",
                                                                  "status": "ASSOCIATED",
                                                                  "firstSeenTime": 1487771895967,
                                                                  "wgbStatus": "REGULARCLIENT",
                                                                  "vendor": "Samsung Electronics", "vlan": "10",
                                                                  "ccxMSVersion": "V1", "nacState": "ACCESS",
                                                                  "webSecurity": "DISABLED", "policyType": "NA",
                                                                  "associationTime": 1488178285705, "ifIndex": 0,
                                                                  "apIpAddress": {"address": "1.2.2.2"}, "@id": 4110106,
                                                                  "eapType": "UNNOWN",
                                                                  "mobilityStatus": "EXPORTANCHORED", "ipType": "IPV4",
                                                                  "ipAddress": {"address": "10.2.2.2"}, "apSlotId": 0,
                                                                  "encryptionCypher": "NONE",
                                                                  "apMacAddress": "FF:FF:FF:FF:FF:FF",
                                                                  "ccxFSVersion": "V1", "clientApMode": "UNKNOWN",
                                                                  "deviceIpAddress": {"address": "172.2.2.2"},
                                                                  "clientAclApplied": "NA"},
                                             "@url": "https://ciscoprime.com/webacs/api/v2/data/ClientDetails/4110106",
                                             "@type": "ClientDetails", "@dtoType": "clientDetailsDTO"},
                                            {"clientDetailsDTO": {"apName": "nzcfd-rotooff-ap2", "deviceName": "cntl-1",
                                                                  "updateTime": 1490290426910,
                                                                  "protocol": "DOT11N2_4GNZ",
                                                                  "clientInterface": "management",
                                                                  "macAddress": "FF:FF:FF:FF:FF:FF",
                                                                  "ccxLSVersion": "V1", "policyTypeStatus": "NOMETHOD",
                                                                  "wgbMacAddress": "00:00:00:00:00:00",
                                                                  "wepState": "DISABLED", "ccxVersion": "UNSUPPORTED",
                                                                  "speed": "UNKNOWN", "@displayName": "4110106",
                                                                  "hreapLocallyAuthenticated": "2", "ssid": "Guest",
                                                                  "clientAaaOverrideAclApplied": "NA",
                                                                  "ccxVSVersion": "V1",
                                                                  "securityPolicyStatus": "FAILED",
                                                                  "authenticationAlgorithm": "OPENSYSTEM",
                                                                  "connectionType": "LIGHTWEIGHTWIRELESS",
                                                                  "postureStatus": "UNKNOWN", "deviceType": "none",
                                                                  "location": "Root Area", "wiredClientType": "NA",
                                                                  "status": "ASSOCIATED",
                                                                  "firstSeenTime": 1487771895967,
                                                                  "wgbStatus": "REGULARCLIENT",
                                                                  "vendor": "Samsung Electronics", "vlan": "10",
                                                                  "ccxMSVersion": "V1", "nacState": "ACCESS",
                                                                  "webSecurity": "DISABLED", "policyType": "NA",
                                                                  "associationTime": 1488178285705, "ifIndex": 0,
                                                                  "apIpAddress": {"address": "1.2.2.2"}, "@id": 4110107,
                                                                  "eapType": "UNNOWN",
                                                                  "mobilityStatus": "EXPORTANCHORED", "ipType": "IPV4",
                                                                  "ipAddress": {"address": "10.2.2.2"}, "apSlotId": 0,
                                                                  "encryptionCypher": "NONE",
                                                                  "apMacAddress": "FF:FF:FF:FF:FF:FF",
                                                                  "ccxFSVersion": "V1", "clientApMode": "UNKNOWN",
                                                                  "deviceIpAddress": {"address": "172.2.2.2"},
                                                                  "clientAclApplied": "NA"},
                                             "@url": "https://ciscoprime.com/webacs/api/v2/data/ClientDetails/4110106",
                                             "@type": "ClientDetails", "@dtoType": "clientDetailsDTO"}]}}

clientlist = my_response['queryResponse'].get('entity', [])

# Initialize my empty dict
formatted_clients = {}

if clientlist:
    for client in clientlist:
        # Initialize an empty client dict.
        client_dict = {}
        # Validate clientDetailsDTO
        if "clientDetailsDTO" in client:
            client_id = client['clientDetailsDTO'].get('@id', '')
            if not client_id:
                # Let's bail out if this doesn't exist
                continue
            for key in client['clientDetailsDTO'].keys():
                # iterate through the keys, and add them to the client dict
                if key == "@id":
                    # we don't need the client id key inside the dict
                    continue
                client_dict[key] = client['clientDetailsDTO'][key]
            # append the client_dict to formatted list
            formatted_clients[client_id] = client_dict

print(json.dumps(formatted_clients, indent=4))

Result:

{
    "4110106": {
        "firstSeenTime": 1487771895967,
        "status": "ASSOCIATED",
        "@displayName": "4110106",
        "clientAaaOverrideAclApplied": "NA",
        "mobilityStatus": "EXPORTANCHORED",
        "ccxFSVersion": "V1",
        "deviceType": "none",
        "ccxMSVersion": "V1",
        "speed": "UNKNOWN",
        "location": "Root Area",
        "ipType": "IPV4",
        "wepState": "DISABLED",
        "eapType": "UNNOWN",
        "wiredClientType": "NA",
        "hreapLocallyAuthenticated": "2",
        "apMacAddress": "FF:FF:FF:FF:FF:FF",
        "ccxVersion": "UNSUPPORTED",
        "webSecurity": "DISABLED",
        "wgbMacAddress": "00:00:00:00:00:00",
        "encryptionCypher": "NONE",
        "nacState": "ACCESS",
        "macAddress": "FF:FF:FF:FF:FF:FF",
        "securityPolicyStatus": "FAILED",
        "ssid": "Guest",
        "deviceIpAddress": {
            "address": "172.2.2.2"
        },
        "clientAclApplied": "NA",
        "ccxVSVersion": "V1",
        "ifIndex": 0,
        "clientApMode": "UNKNOWN",
        "apIpAddress": {
            "address": "1.2.2.2"
        },
        "connectionType": "LIGHTWEIGHTWIRELESS",
        "vlan": "10",
        "ccxLSVersion": "V1",
        "apSlotId": 0,
        "clientInterface": "management",
        "protocol": "DOT11N2_4GNZ",
        "authenticationAlgorithm": "OPENSYSTEM",
        "vendor": "Samsung Electronics",
        "updateTime": 1490290426910,
        "deviceName": "cntl-1",
        "apName": "nzcfd-rotooff-ap2",
        "ipAddress": {
            "address": "10.2.2.2"
        },
        "wgbStatus": "REGULARCLIENT",
        "postureStatus": "UNKNOWN",
        "policyTypeStatus": "NOMETHOD",
        "associationTime": 1488178285705,
        "policyType": "NA"
    },
    "4110107": {
        "firstSeenTime": 1487771895967,
        "status": "ASSOCIATED",
        "@displayName": "4110106",
        "clientAaaOverrideAclApplied": "NA",
        "mobilityStatus": "EXPORTANCHORED",
        "ccxFSVersion": "V1",
        "deviceType": "none",
        "ccxMSVersion": "V1",
        "speed": "UNKNOWN",
        "location": "Root Area",
        "ipType": "IPV4",
        "wepState": "DISABLED",
        "eapType": "UNNOWN",
        "wiredClientType": "NA",
        "hreapLocallyAuthenticated": "2",
        "apMacAddress": "FF:FF:FF:FF:FF:FF",
        "ccxVersion": "UNSUPPORTED",
        "webSecurity": "DISABLED",
        "wgbMacAddress": "00:00:00:00:00:00",
        "encryptionCypher": "NONE",
        "nacState": "ACCESS",
        "macAddress": "FF:FF:FF:FF:FF:FF",
        "securityPolicyStatus": "FAILED",
        "ssid": "Guest",
        "deviceIpAddress": {
            "address": "172.2.2.2"
        },
        "clientAclApplied": "NA",
        "ccxVSVersion": "V1",
        "ifIndex": 0,
        "clientApMode": "UNKNOWN",
        "apIpAddress": {
            "address": "1.2.2.2"
        },
        "connectionType": "LIGHTWEIGHTWIRELESS",
        "vlan": "10",
        "ccxLSVersion": "V1",
        "apSlotId": 0,
        "clientInterface": "management",
        "protocol": "DOT11N2_4GNZ",
        "authenticationAlgorithm": "OPENSYSTEM",
        "vendor": "Samsung Electronics",
        "updateTime": 1490290426910,
        "deviceName": "cntl-1",
        "apName": "nzcfd-rotooff-ap2",
        "ipAddress": {
            "address": "10.2.2.2"
        },
        "wgbStatus": "REGULARCLIENT",
        "postureStatus": "UNKNOWN",
        "policyTypeStatus": "NOMETHOD",
        "associationTime": 1488178285705,
        "policyType": "NA"
    }
}

In you exact context this code will work with the fields you want:

import json
import requests 

url = 'https://ciscoprime.com/webacs/api/v2/data/ClientDetails.json?.nocount=True&.maxResults=2&.firstResult=0&.full=true'
get_response = requests.get(url, verify=False)
response = json.dumps(get_response.json())
resp = json.loads(response)


clientlist = resp['queryResponse'].get('entity', [])

# Initialize my empty dict
formatted_clients = {}
search_keys = ["associationTime", "clientInterface", "connectionType", "deviceIpAddress", "deviceName", "firstSeenTime", "macAddress", "updateTime", "vendor", "vlan", "vlanName"]
if clientlist:
    for client in clientlist:
        # Initialize an empty client dict.
        client_dict = {}
        # Validate clientDetailsDTO
        if "clientDetailsDTO" in client:
            client_id = client['clientDetailsDTO'].get('@id', '')
            if not client_id:
                # Let's bail out if this doesn't exist
                continue
            for key in client['clientDetailsDTO'].keys():
                # iterate through the keys, and add them to the client dict
                if key in search_keys:
                    client_dict[key] = client['clientDetailsDTO'][key]
            # append the client_dict to formatted list
            formatted_clients[client_id] = client_dict

print(json.dumps(formatted_clients, indent=4))
Improve this answer
Sign up to request clarification or add additional context in comments.

Comments

0

You can update your client dictionary something like this : 

clients = []
for data in resp['queryResponse']['entity'] : 
    clientID = data.get("clientDetailsDTO", {}).get("@id")
    associationTime = data.get("clientDetailsDTO", {}).get("associationTime")
    clientInterface = data.get("clientDetailsDTO", {}).get("clientInterface")
    connectionType = data.get("clientDetailsDTO", {}).get("connectionType")
    deviceIpAddress = data.get("clientDetailsDTO", {}).get("deviceIpAddress")
    deviceName = data.get("clientDetailsDTO", {}).get("deviceName")
    firstSeenTime = data.get("clientDetailsDTO", {}).get("firstSeenTime")
    ifIndex = data.get("clientDetailsDTO", {}).get("ifIndex")
    ipAddress = data.get("clientDetailsDTO", {}).get("ipAddress")
    macAddress = data.get("clientDetailsDTO", {}).get("macAddress")
    updateTime = data.get("clientDetailsDTO", {}).get("updateTime")
    vendor = data.get("clientDetailsDTO", {}).get("vendor")
    vlan = data.get("clientDetailsDTO", {}).get("vlan")
    vlanName = data.get("clientDetailsDTO", {}).get("vlanName")

    clients.append({
        'ClientID' : clientID, 
        'value' :
        {
            'associationTime' : associationTime, 
            'vlanName' : vlanName,
            'connectionType' : connectionType
        }
    })

json.dumps(clients)
Improve this answer

7 Comments

that is what i was initially looking at doing, however the dictionary gets overwritten. So if I'm looping through all of the identity's so for testing i'm just doing two. It only does the last one as it is getting overwritten. I obviously need to take this part out of the loop, But i'm still learning so i'm not sure how to do that yet. This also does not follow the data structure that i'm looking for.
Firstly, clients dictionary is declared ouside od the loop so its data won't get overwritten. Secondly, you don not need to use i explicitly and use for loop.
This looks great, and should work. But i'm getting a syntax error from line 24 or 25 but the syntax looks correct.
What is the traceback exactly ?
# python primeapi2.py File "primeapi2.py", line 25 }) ^ SyntaxError: invalid syntax
|

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.