0 
{
    "Logging": {
        "LogLevel": {
            "Default": "Information",
            "Microsoft": "Warning",
            "Microsoft.Hosting.Lifetime": "Information",
            "Microsoft.AspNetCore": "Warning",
            "System.Net.Http.HttpClient.Default.ClientHandler": "Warning",
            "System.Net.Http.HttpClient.Default.LogicalHandler": "Warning"
        }
    },
    "AllowedHosts": "*",
    "AutomaticTransferOptions": {
        "DateOffsetForDirectoriesInDays": -1,
        "DateOffsetForPortfoliosInDays": -3,
        "Clause": {
            "Item1": "1"
        }    
    },
    "Authentication": {
        "ApiKeys": [
            {
                "Key": "AB8E5976-2A7C-4EEE-92C1-7B0B4DC840F6",
                "OwnerName": "Cron job",
                "Claims": [
                    {
                        "Type": "http://schemas.microsoft.com/ws/2008/06/identity/claims/role",
                        "Value": "StressTestManager"
                    }
                ]
            },
            {
                "Key": "B11D4F27-483A-4234-8EC7-CA121712D5BE",
                "OwnerName": "Test admin",
                "Claims": [
                    {
                        "Type": "http://schemas.microsoft.com/ws/2008/06/identity/claims/role",
                        "Value": "StressTestAdmin"
                    },
                    {
                        "Type": "http://schemas.microsoft.com/ws/2008/06/identity/claims/role",
                        "Value": "TestManager"
                    }
                ]
            },
            {
                "Key": "EBF98F2E-555E-4E66-9D77-5667E0AA1B54",
                "OwnerName": "Test manager",
                "Claims": [
                    {
                        "Type": "http://schemas.microsoft.com/ws/2008/06/identity/claims/role",
                        "Value": "TestManager"
                    }
                ]
            }
        ],
        "LDAP": {
            "Domain": "domain.local",
            "MachineAccountName": "Soft13",
            "MachineAccountPassword": "vixuUEY7884*",
            "EnableLdapClaimResolution": true
        }
    },
    "Authorization": {
        "Permissions": {
            "Roles": [
                {
                    "Role": "TestAdmin",
                    "Permissions": [
                        "transfers.create",
                        "bindings.create"
                    ]
                },
                {
                    "Role": "TestManager",
                    "Permissions": [
                        "transfers.create"
                    ]
                }
            ]
        }
    }
}

I have JSON above and need to parse it with output like this

Logging__LogLevel__Default
Authentication__ApiKeys__0__Claims__0__Type

Everything is ok, but I always get some strings with this output

Authentication__ApiKeys__0__Key
Authentication__ApiKeys__0__OwnerName
Authentication__ApiKeys__0__Claims__0__Type
Authentication__ApiKeys__0__Claims__0__Value
Authentication__ApiKeys__0__Claims__0
Authentication__ApiKeys__2
Authorization__Permissions__Roles__0__Role
Authorization__Permissions__Roles__0__Permissions__1
Authorization__Permissions__Roles__1__Role
Authorization__Permissions__Roles__1__Permissions__0
Authorization__Permissions__Roles__1

Why does my code adds not full strings like

Authentication__ApiKeys__0__Claims__0
Authentication__ApiKeys__2
Authorization__Permissions__Roles__1

And why it doesn't print every value from

Authorization__Permissions__Roles__0__Permissions__*

and from

Authorization__Permissions__Roles__1__Permissions__*

I have this code in python3:

def checkdepth(sub_key, variable):
    delmt = '__'
    for item in sub_key:
        try:
            if isinstance(sub_key[item], dict):
                sub_variable = variable + delmt + item
                checkdepth(sub_key[item], sub_variable)
        except TypeError:
            continue

        if isinstance(sub_key[item], list):
            sub_variable = variable + delmt + item
            for it in sub_key[item]:
                sub_variable = variable + delmt + item + delmt + str(sub_key[item].index(it))
                checkdepth(it, sub_variable)
            print(sub_variable)

        if isinstance(sub_key[item], int) or isinstance(sub_key[item], str):
                sub_variable = variable + delmt + item
                print (sub_variable)

for key in data:   
    if type(data[key]) is str:
       print(key + '=' +str(data[key]))
    else:
      variable = key
      checkdepth(data[key], variable)

I know that the problem in block where I process list data type, but I don't know where is the problem exactly

Improve this question
2
  • Please post what the expected result is for Authorization__Permissions__Roles__0__Permissions__*. What should it be showing? Also I ran your code from what you've posted and I don't have the issue of "not full strings". So post all your code cause it could be somewhere else in your code. Commented Dec 27, 2020 at 18:52
  • It's my full code. I meant that this code adds strings like Authentication__ApiKeys__2 But I need just full strings with all levels of json nest, for example Authentication__ApiKeys__0__Claims__0__Value I expect that this code will print string for every element from Permissions list: Authorization__Permissions__Roles__0__Permissions__0 Authorization__Permissions__Roles__1__Permissions__0 Authorization__Permissions__Roles__1__Permissions__1 Commented Dec 27, 2020 at 19:02

3 Answers 3

Reset to default
1

Use a recursive generator:

import json

with open('input.json') as f:
    data = json.load(f)

def strkeys(data):
    if isinstance(data,dict):
        for k,v in data.items():
            for item in strkeys(v):
                yield f'{k}__{item}' if item else k
    elif isinstance(data,list):
        for i,v in enumerate(data):
            for item in strkeys(v):
                yield f'{i}__{item}' if item else str(i)
    else:
        yield None # termination condition, not a list or dict

for s in strkeys(data):
    print(s)

Output:

Logging__LogLevel__Default
Logging__LogLevel__Microsoft
Logging__LogLevel__Microsoft.Hosting.Lifetime
Logging__LogLevel__Microsoft.AspNetCore
Logging__LogLevel__System.Net.Http.HttpClient.Default.ClientHandler
Logging__LogLevel__System.Net.Http.HttpClient.Default.LogicalHandler
AllowedHosts
AutomaticTransferOptions__DateOffsetForDirectoriesInDays
AutomaticTransferOptions__DateOffsetForPortfoliosInDays
AutomaticTransferOptions__Clause__Item1
Authentication__ApiKeys__0__Key
Authentication__ApiKeys__0__OwnerName
Authentication__ApiKeys__0__Claims__0__Type
Authentication__ApiKeys__0__Claims__0__Value
Authentication__ApiKeys__1__Key
Authentication__ApiKeys__1__OwnerName
Authentication__ApiKeys__1__Claims__0__Type
Authentication__ApiKeys__1__Claims__0__Value
Authentication__ApiKeys__1__Claims__1__Type
Authentication__ApiKeys__1__Claims__1__Value
Authentication__ApiKeys__2__Key
Authentication__ApiKeys__2__OwnerName
Authentication__ApiKeys__2__Claims__0__Type
Authentication__ApiKeys__2__Claims__0__Value
Authentication__LDAP__Domain
Authentication__LDAP__MachineAccountName
Authentication__LDAP__MachineAccountPassword
Authentication__LDAP__EnableLdapClaimResolution
Authorization__Permissions__Roles__0__Role
Authorization__Permissions__Roles__0__Permissions__0
Authorization__Permissions__Roles__0__Permissions__1
Authorization__Permissions__Roles__1__Role
Authorization__Permissions__Roles__1__Permissions__0
Improve this answer
Sign up to request clarification or add additional context in comments.

1 Comment

wonderful code, would be great to understand how it works ) thank you!
1

Using json_flatten this can be converted to pandas, but it's not clear if that's what you want. Also, when you do convert it can use df.iloc[0] to see why each column is being provided (ie you see the value for that key).

Note: you need to pass a list so I just wrapped your json above in [].

# https://github.com/amirziai/flatten
dic = your json from above
dic =[dic] # put it in a list
dic_flattened = (flatten(d, '__') for d in dic) # add your delimiter
df = pd.DataFrame(dic_flattened)

df.iloc[0]

Logging__LogLevel__Default                                                                                    Information
Logging__LogLevel__Microsoft                                                                                      Warning
Logging__LogLevel__Microsoft.Hosting.Lifetime                                                                 Information
Logging__LogLevel__Microsoft.AspNetCore                                                                           Warning
Logging__LogLevel__System.Net.Http.HttpClient.Default.ClientHandler                                               Warning
Logging__LogLevel__System.Net.Http.HttpClient.Default.LogicalHandler                                              Warning
AllowedHosts                                                                                                            *
AutomaticTransferOptions__DateOffsetForDirectoriesInDays                                                               -1
AutomaticTransferOptions__DateOffsetForPortfoliosInDays                                                                -3
AutomaticTransferOptions__Clause__Item1                                                                                 1
Authentication__ApiKeys__0__Key                                                      AB8E5976-2A7C-4EEE-92C1-7B0B4DC840F6
Authentication__ApiKeys__0__OwnerName                                                                            Cron job
Authentication__ApiKeys__0__Claims__0__Type                             http://schemas.microsoft.com/ws/2008/06/identi...
Authentication__ApiKeys__0__Claims__0__Value                                                            StressTestManager
Authentication__ApiKeys__1__Key                                                      B11D4F27-483A-4234-8EC7-CA121712D5BE
Authentication__ApiKeys__1__OwnerName                                                                          Test admin
Authentication__ApiKeys__1__Claims__0__Type                             http://schemas.microsoft.com/ws/2008/06/identi...
Authentication__ApiKeys__1__Claims__0__Value                                                              StressTestAdmin
Authentication__ApiKeys__1__Claims__1__Type                             http://schemas.microsoft.com/ws/2008/06/identi...
Authentication__ApiKeys__1__Claims__1__Value                                                                  TestManager
Authentication__ApiKeys__2__Key                                                      EBF98F2E-555E-4E66-9D77-5667E0AA1B54
Authentication__ApiKeys__2__OwnerName                                                                        Test manager
Authentication__ApiKeys__2__Claims__0__Type                             http://schemas.microsoft.com/ws/2008/06/identi...
Authentication__ApiKeys__2__Claims__0__Value                                                                  TestManager
Authentication__LDAP__Domain                                                                                 domain.local
Authentication__LDAP__MachineAccountName                                                                           Soft13
Authentication__LDAP__MachineAccountPassword                                                                 vixuUEY7884*
Authentication__LDAP__EnableLdapClaimResolution                                                                      true
Authorization__Permissions__Roles__0__Role                                                                      TestAdmin
Authorization__Permissions__Roles__0__Permissions__0                                                     transfers.create
Authorization__Permissions__Roles__0__Permissions__1                                                      bindings.create
Authorization__Permissions__Roles__1__Role                                                                    TestManager
Authorization__Permissions__Roles__1__Permissions__0                                                     transfers.create
Improve this answer

Comments

1

Ok, I looked at your code and it's hard to follow. You're variable and function names are not easy to understand their purpose. Which is fine cause everyone has to learn best practice and all the little tips and tricks in python. So hopefully I can help you out.

  1. You have a recursive-ish function. Which is definingly the best way to handle a situation like this. However your code is part recursive and part not. If you go recursive to solve a problem you have to go 100% recursive.

  2. Also the only time you should print in a recursive function is for debugging. Recursive functions should have an object that is passed down the function and gets appended to or altered and then passed back once it gets to the end of the recursion.

  3. When you get a problem like this, think about which data you actually need or care about. In this problem we don't care about the values that are stored in the object, we just care about the keys. So we should write code that doesn't even bother looking at the value of something except to determine its type.

Here is some code I wrote up that should work for what you're wanting to do. But take note that because I did purely a recursive function my code base is small. Also my function uses a list that is passed around and added to and then at the end I return it so that we can use it for whatever we need. If you have questions just comment on this question and I'll answer the best I can.

def convert_to_delimited_keys(obj, parent_key='', delimiter='__', keys_list=None):
    if keys_list is None: keys_list = []

    if isinstance(obj, dict):
        for k in obj:
            convert_to_delimited_keys(obj[k], delimiter.join((parent_key, str(k))), delimiter, keys_list)
    elif isinstance(obj, list):
        for i, _ in enumerate(obj):
            convert_to_delimited_keys(obj[i], delimiter.join((parent_key, str(i))), delimiter, keys_list)
    else:
        # Append to list, but remove the leading delimiter due to string.join
        keys_list.append(parent_key[len(delimiter):])
    return keys_list

for item in convert_to_delimited_keys(data):
    print(item)
Improve this answer

4 Comments

There's a bug if you run the function more than once. keys_list is a mutable default value and isn't reset on future runs.
Yes, if you create a list and pass it into the function for keys_list it will work. The default is there only if you plan to run it once.
Use keys_list=None and in the body if keys_list is None: keys_list = [] to fix.
Yea that is probably the better way to handle it.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.