0

I want to write a python script that loops through several JSON files and checks whether an attribute(KEY:VALUE) as shown below

"grp_farmerreg/farmerdetails/farmermobile": "0756625759",

exists and if not check for hh_id as shown belown

"grp_farmerdts/hh_id": "0753693707",

after confirming availability of hh_id and absence of farmemobile i would like to add

"grp_farmerreg/farmerdetails/farmermobile":(number from hh_id)

Here is an example of my JSON file without my farmermobile

{
    "_notes": [], 
    "_bamboo_dataset_id": "", 
    "_tags": [], 
    "regdate": "2018-10-17", 
    "gpsloc": "-9.109567398208265 32.947977763771334 1594.4399713018483 8.0", 
    "_xform_id_string": "ADGG-TZA-REG02-20181008", 
    "meta/instanceID": "uuid:4d5372ed-34f0-4ac3-b151-f4f57b746ad4", 
    "_duration": "", 
    "grp_calfreg/rpt_b_calvedets": [
        {
            "grp_calfreg/rpt_b_calvedets/rpt_b_calvedets_layout/calfsex": "2", 
            "grp_calfreg/rpt_b_calvedets/calv_alive_geninfo/calfname": "Nill", 
            "grp_calfreg/rpt_b_calvedets/calc_calfsireid": "TNZ000404011991", 
            "grp_calfreg/rpt_b_calvedets/calfSire_Owner": "3", 
            "grp_calfreg/rpt_b_calvedets/calfsirebrthyrknown": "0", 
            "grp_calfreg/rpt_b_calvedets/rpt_b_calvedets_layout/easecalv": "2", 
            "grp_calfreg/rpt_b_calvedets/calv_alive_geninfo/calftagimage": "1539766349136.jpg", 
            "grp_calfreg/rpt_b_calvedets/calfsiredetails/calfSire_secBreed": "2", 
            "grp_calfreg/rpt_b_calvedets/calv_alive_layout/calvmeasures/calfhgirth": "100.0", 
            "grp_calfreg/rpt_b_calvedets/calfsiredetails/calfSire_ShortName": "Nill", 
            "grp_calfreg/rpt_b_calvedets/calv_alive_geninfo/calftagprefix": "00040401", 
            "grp_calfreg/rpt_b_calvedets/calc_calfid": "TNZ000404014693", 
            "grp_calfreg/rpt_b_calvedets/grp_calfid/Confirm_CalfID": "OK", 
            "grp_calfreg/rpt_b_calvedets/calfSire_OwnerFarmer": "Nill", 
            "grp_calfreg/rpt_b_calvedets/calfSire_TagID": "000404011991", 
            "grp_calfreg/rpt_b_calvedets/calv_alive_layout/calvmeasures/calfweight": "80.0", 
            "grp_calfreg/rpt_b_calvedets/grp_calfid/Calf_ID": "TNZ000404014693", 
            "grp_calfreg/rpt_b_calvedets/feedmthd/feedmth": "-66", 
            "grp_calfreg/rpt_b_calvedets/calv_alive_layout/calvmeasures/calfbodyscore": "4.0", 
            "grp_calfreg/rpt_b_calvedets/calfsirentregstrd": "1", 
            "grp_calfreg/rpt_b_calvedets/calfsirebrthdateknown": "0", 
            "grp_calfreg/rpt_b_calvedets/calfsiredetails/calfSire_mainComp": "5", 
            "grp_calfreg/rpt_b_calvedets/grp_calfsireid/Confirm_CalfsireID": "OK", 
            "grp_calfreg/rpt_b_calvedets/grp_calfsireid/Calfsire_ID": "TNZ000404011991", 
            "grp_calfreg/rpt_b_calvedets/calv_alive_layout/fmlcalfuse/intuse": "1", 
            "grp_calfreg/rpt_b_calvedets/rpt_b_calvedets_layout/calfsiretype": "1", 
            "grp_calfreg/rpt_b_calvedets/rpt_b_calvedets_layout/calvtype": "1", 
            "grp_calfreg/rpt_b_calvedets/rpt_b_calvedets_layout/birthtyp": "1", 
            "grp_calfreg/rpt_b_calvedets/rpt_b_calvedets_layout/calvdatealv": "2018-05-09", 
            "grp_calfreg/rpt_b_calvedets/rpt_b_calvedets_layout/damid": "TZN000404014225", 
            "grp_calfreg/rpt_b_calvedets/calv_alive_geninfo/calftagsec": "4693", 
            "grp_calfreg/rpt_b_calvedets/calfsiredetails/calfSire_Country": "Unknown", 
            "grp_calfreg/rpt_b_calvedets/damidyesno": "1", 
            "grp_calfreg/rpt_b_calvedets/calv_alive_geninfo/calfbodyimage": "1539766363236.jpg", 
            "grp_calfreg/rpt_b_calvedets/calfsiredetails/calfSire_mainBreed": "2", 
            "grp_calfreg/rpt_b_calvedets/calfsiredetails/calfSire_HerdBkRegNo": "Nill", 
            "grp_calfreg/rpt_b_calvedets/feedmthd/feedmthoth": "Supliment", 
            "grp_calfreg/rpt_b_calvedets/calv_alive_layout/calfdef/calfdeformities": "1", 
            "grp_calfreg/rpt_b_calvedets/calfsireregisterd": "0", 
            "grp_calfreg/rpt_b_calvedets/calv_alive_geninfo/calfcolor": "Black and white", 
            "grp_calfreg/rpt_b_calvedets/calv_alive_layout/calfdef/calfwightknown": "1", 
            "grp_calfreg/rpt_b_calvedets/calfsiredetails/calfSire_RegisteredName": "Nill"
        }
    ], 
    "grp_farmerdts/region": "1007", 
    "_geolocation": [
        -9.109567398208265, 
        32.947977763771334
    ], 
    "datacollid": "0758990688", 
    "_status": "submitted_via_web", 
    "farmerregistrd": "1", 
    "deviceid": "353422071089618", 
    "start_time": "2018-10-17T11:45:49.406+03", 
    "_uuid": "4d5372ed-34f0-4ac3-b151-f4f57b746ad4", 
    "grp_farmerdts/village": "4864", 
    "enumtype": "2", 
    "_submitted_by": null, 
    "formhub/uuid": "678f27aca56149a391069c33019a34fc", 
    "grp_farmerdts/hh_id": "0753693707", 
    "_id": 626391, 
    "regactivity": "5", 
    "_submission_time": "2018-10-22T03:18:24", 
    "_version": "20181008", 
    "_attachments": [
        {
            "mimetype": "image/jpeg", 
            "download_url": "http://localhost:8000/media/adggtnz/attachments/1539766349136.jpg", 
            "filename": "adggtnz/attachments/1539766349136.jpg", 
            "instance": 626391, 
            "id": 102154, 
            "xform": 3443
        }, 
        {
            "mimetype": "image/jpeg", 
            "download_url": "http://localhost:8000/media/adggtnz/attachments/1539766363236.jpg", 
            "filename": "adggtnz/attachments/1539766363236.jpg", 
            "instance": 626391, 
            "id": 102153, 
            "xform": 3443
        }
    ], 
    "end_time": "2018-10-17T11:53:41.354+03", 
    "grp_farmerdts/country": "2", 
    "_userform_id": "adggtnz_ADGG-TZA-REG02-20181008", 
    "grp_farmerdts/ward": "1807", 
    "grp_farmerdts/district": "1037"
}

i would like to put my output on a different folder to show the original and the updated JSON file

expected output:

 {
        "_notes": [], 
        "_bamboo_dataset_id": "", 
        "_tags": [], 
        "regdate": "2018-10-17", 
        "gpsloc": "-9.109567398208265 32.947977763771334 1594.4399713018483 8.0", 
        "_xform_id_string": "ADGG-TZA-REG02-20181008", 
        "meta/instanceID": "uuid:4d5372ed-34f0-4ac3-b151-f4f57b746ad4", 
        "_duration": "", 
        "grp_calfreg/rpt_b_calvedets": [
            {
                "grp_calfreg/rpt_b_calvedets/rpt_b_calvedets_layout/calfsex": "2", 
                "grp_calfreg/rpt_b_calvedets/calv_alive_geninfo/calfname": "Nill", 
                "grp_calfreg/rpt_b_calvedets/calc_calfsireid": "TNZ000404011991", 
                "grp_calfreg/rpt_b_calvedets/calfSire_Owner": "3", 
                "grp_calfreg/rpt_b_calvedets/calfsirebrthyrknown": "0", 
                "grp_calfreg/rpt_b_calvedets/rpt_b_calvedets_layout/easecalv": "2", 
                "grp_calfreg/rpt_b_calvedets/calv_alive_geninfo/calftagimage": "1539766349136.jpg", 
                "grp_calfreg/rpt_b_calvedets/calfsiredetails/calfSire_secBreed": "2", 
                "grp_calfreg/rpt_b_calvedets/calv_alive_layout/calvmeasures/calfhgirth": "100.0", 
                "grp_calfreg/rpt_b_calvedets/calfsiredetails/calfSire_ShortName": "Nill", 
                "grp_calfreg/rpt_b_calvedets/calv_alive_geninfo/calftagprefix": "00040401", 
                "grp_calfreg/rpt_b_calvedets/calc_calfid": "TNZ000404014693", 
                "grp_calfreg/rpt_b_calvedets/grp_calfid/Confirm_CalfID": "OK", 
                "grp_calfreg/rpt_b_calvedets/calfSire_OwnerFarmer": "Nill", 
                "grp_calfreg/rpt_b_calvedets/calfSire_TagID": "000404011991", 
                "grp_calfreg/rpt_b_calvedets/calv_alive_layout/calvmeasures/calfweight": "80.0", 
                "grp_calfreg/rpt_b_calvedets/grp_calfid/Calf_ID": "TNZ000404014693", 
                "grp_calfreg/rpt_b_calvedets/feedmthd/feedmth": "-66", 
                "grp_calfreg/rpt_b_calvedets/calv_alive_layout/calvmeasures/calfbodyscore": "4.0", 
                "grp_calfreg/rpt_b_calvedets/calfsirentregstrd": "1", 
                "grp_calfreg/rpt_b_calvedets/calfsirebrthdateknown": "0", 
                "grp_calfreg/rpt_b_calvedets/calfsiredetails/calfSire_mainComp": "5", 
                "grp_calfreg/rpt_b_calvedets/grp_calfsireid/Confirm_CalfsireID": "OK", 
                "grp_calfreg/rpt_b_calvedets/grp_calfsireid/Calfsire_ID": "TNZ000404011991", 
                "grp_calfreg/rpt_b_calvedets/calv_alive_layout/fmlcalfuse/intuse": "1", 
                "grp_calfreg/rpt_b_calvedets/rpt_b_calvedets_layout/calfsiretype": "1", 
                "grp_calfreg/rpt_b_calvedets/rpt_b_calvedets_layout/calvtype": "1", 
                "grp_calfreg/rpt_b_calvedets/rpt_b_calvedets_layout/birthtyp": "1", 
                "grp_calfreg/rpt_b_calvedets/rpt_b_calvedets_layout/calvdatealv": "2018-05-09", 
                "grp_calfreg/rpt_b_calvedets/rpt_b_calvedets_layout/damid": "TZN000404014225", 
                "grp_calfreg/rpt_b_calvedets/calv_alive_geninfo/calftagsec": "4693", 
                "grp_calfreg/rpt_b_calvedets/calfsiredetails/calfSire_Country": "Unknown", 
                "grp_calfreg/rpt_b_calvedets/damidyesno": "1", 
                "grp_calfreg/rpt_b_calvedets/calv_alive_geninfo/calfbodyimage": "1539766363236.jpg", 
                "grp_calfreg/rpt_b_calvedets/calfsiredetails/calfSire_mainBreed": "2", 
                "grp_calfreg/rpt_b_calvedets/calfsiredetails/calfSire_HerdBkRegNo": "Nill", 
                "grp_calfreg/rpt_b_calvedets/feedmthd/feedmthoth": "Supliment", 
                "grp_calfreg/rpt_b_calvedets/calv_alive_layout/calfdef/calfdeformities": "1", 
                "grp_calfreg/rpt_b_calvedets/calfsireregisterd": "0", 
                "grp_calfreg/rpt_b_calvedets/calv_alive_geninfo/calfcolor": "Black and white", 
                "grp_calfreg/rpt_b_calvedets/calv_alive_layout/calfdef/calfwightknown": "1", 
                "grp_calfreg/rpt_b_calvedets/calfsiredetails/calfSire_RegisteredName": "Nill"
            }
        ], 
        "grp_farmerdts/region": "1007", 
        "_geolocation": [
            -9.109567398208265, 
            32.947977763771334
        ], 
        "datacollid": "0758990688", 
        "_status": "submitted_via_web", 
        "farmerregistrd": "1", 
        "deviceid": "353422071089618", 
        "start_time": "2018-10-17T11:45:49.406+03", 
        "_uuid": "4d5372ed-34f0-4ac3-b151-f4f57b746ad4", 
        "grp_farmerdts/village": "4864", 
        "enumtype": "2", 
        "_submitted_by": null, 
        "formhub/uuid": "678f27aca56149a391069c33019a34fc", 
        "grp_farmerdts/hh_id": "0753693707", 
        "grp_farmerreg/farmerdetails/farmermobile": "0753693707",
        "_id": 626391, 
        "regactivity": "5", 
        "_submission_time": "2018-10-22T03:18:24", 
        "_version": "20181008", 
        "_attachments": [
            {
                "mimetype": "image/jpeg", 
                "download_url": "http://localhost:8000/media/adggtnz/attachments/1539766349136.jpg", 
                "filename": "adggtnz/attachments/1539766349136.jpg", 
                "instance": 626391, 
                "id": 102154, 
                "xform": 3443
            }, 
            {
                "mimetype": "image/jpeg", 
                "download_url": "http://localhost:8000/media/adggtnz/attachments/1539766363236.jpg", 
                "filename": "adggtnz/attachments/1539766363236.jpg", 
                "instance": 626391, 
                "id": 102153, 
                "xform": 3443
            }
        ], 
        "end_time": "2018-10-17T11:53:41.354+03", 
        "grp_farmerdts/country": "2", 
        "_userform_id": "adggtnz_ADGG-TZA-REG02-20181008", 
        "grp_farmerdts/ward": "1807", 
        "grp_farmerdts/district": "1037"
    }

here is what i have tried:

import json
import os

json_dir="/new/20180116/"
json_dir_processed="/new/20180116updated/"
for json_file in os.listdir(json_dir):
    if json_file.endswith(".json"):
        processed_json = "%s%s" % (json_dir_processed, json_file)
        json_file = json_dir + json_file
        print "Processing %s -> %s" % (json_file, processed_json)
        with open(json_file, 'r') as f:
            json_data = json.load(f)
            json_data['"grp_farmerreg/farmerdetails/farmermobile": "0753693707"'] = json_data['"grp_farmerdts/hh_id": "0753693707" ']
        with open(processed_json, 'w') as f:
            f.write(json.dumps(json_data, indent=4))
    else:
        print "%s not a JSON file" % json_file

my script results to the following error

Traceback (most recent call last):
  File "new.py", line 15, in <module>
    json_data['"grp_farmerreg/farmerdetails/farmermobile": "0753693707"'] = json_data['"grp_farmerdts/hh_id": "0753693707" ']
KeyError: '"grp_farmerdts/hh_id": "0753693707" '
Improve this question
5
  • What doesn't work with your current attempt? Explain by editing your question. Commented Nov 1, 2018 at 8:30
  • instead of changing value my script is only looping to find this Key:value "grp_farmerdts/hh_id": "0753693707" and replace it with "grp_farmerreg/farmerdetails/farmermobile": "0753693707" Commented Nov 1, 2018 at 8:39
  • @MirieriMogaka Your script would rise KeyError on following line json_data['"grp_farmerreg/farmerdetails/farmermobile": "0753693707"'] = json_data['"grp_farmerdts/hh_id": "0753693707" '] Commented Nov 1, 2018 at 8:44
  • @MirieriMogaka Again, please edit your question and add what's not working there. Not everyone read the comments and they can be deleted. Commented Nov 1, 2018 at 8:48
  • @Evan please help me resolve my problem Commented Nov 1, 2018 at 9:15

2 Answers 2

Reset to default
1

Here is my new script based on input from you guys, it works

import json
import os

json_dir="/media/dmogaka/CENTOS/newforms/regTZA/20181008/"
json_dir_processed="/media/dmogaka/CENTOS/newforms/regTZA/20181008updated/"
for json_file in os.listdir(json_dir):
    if json_file.endswith(".json"):
        processed_json = "%s%s" % (json_dir_processed, json_file)
        json_file = json_dir + json_file
        print "Processing %s -> %s" % (json_file, processed_json)
        with open(json_file, 'r') as f:
            json_data = json.load(f)
            if "grp_farmerreg/farmerdetails/farmermobile" not in json_data:                
                json_data["grp_farmerreg/farmerdetails/farmermobile"] = json_data["grp_farmerdts/hh_id"]
        with open(processed_json, 'w') as f:
            f.write(json.dumps(json_data, indent=4))
    else:
        print "%s not a JSON file" % json_file
Improve this answer
Sign up to request clarification or add additional context in comments.

Comments

0

I'm not 100% sure what you exactly want to do, but the line

json_data['"grp_farmerreg/farmerdetails/farmermobile": "0753693707"'] = json_data['"grp_farmerdts/hh_id": "0753693707" ']

seems to be the problem to me.

You want to first check that the key exists (or equivalently skip one iteration of the loop if it doesn't):

if "grp_farmerreg/farmerdetails/farmermobile" not in json_data:
    continue   # not interested ==> skip to next file

Then, if the key exists, compare it t the value that you desire:

if json_data["grp_farmerreg/farmerdetails/farmermobile"] != "0753693707"
    continue  # not interested => skip to next file

And if you still haven't skipped, you can simply add your new value as

json_data["grp_farmerreg/farmerdetails/farmermobile"] = your new value
Improve this answer

2 Comments

it could be any number, the value "grp_farmerdts/hh_id":"(anything)" this only checks for one value. please edit my script
Sorry, but I still don't understand what you are actually to do. Maybe take a look at stackoverflow.com/help/mcve on how to prepare a small snippet of code that isolates your problem.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.