Add "entry" to JSON File with Python

Question

I need to modify a JSON-File with python. As I'm working with python (and JSON) for the first time, I read some articles about it, but didn't understand it completely.

I managed to import a JSON to python, as some kind of array (or list?).

JSON looks like this:

{
  "sources":[{
    "id":100012630,
    "name":"Activity Login Page",
    "category":"NAM/Activity",
    "automaticDateParsing":true,
    "multilineProcessingEnabled":false,
    "useAutolineMatching":false,
    "forceTimeZone":true,
    "timeZone":"Europe/Brussels",
    "filters":[],
    "cutoffTimestamp":1414364400000,
    "encoding":"UTF-8",
    "pathExpression":"C:\\NamLogs\\nam-login-page.log*",
    "blacklist":[],
    "sourceType":"LocalFile",
    "alive":true
  },{
    "id":100001824,
    "name":"localWinEvent",
    "category":"NAM/OS/EventLog",
    "automaticDateParsing":true,
    "multilineProcessingEnabled":false,
    "useAutolineMatching":false,
    "forceTimeZone":false,
    "filters":[],
    "cutoffTimestamp":1409090400000,
    "encoding":"UTF-8",
    "logNames":["Security","Application","System","Others"],
    "sourceType":"LocalWindowsEventLog",
    "alive":true
  },{
    "id":100001830,
    "name":"localWinPerf",
    "category":"NAM/OS/Perf",
    "automaticDateParsing":false,
    "multilineProcessingEnabled":false,
    "useAutolineMatching":false,
    "forceTimeZone":false,
    "filters":[],
    "cutoffTimestamp":0,
    "encoding":"UTF-8",
    "interval":60000,
    "wmiQueries":[{
      "name":"NAMID Service",
      "query":"SELECT * FROM Win32_PerfRawData_PerfProc_Process WHERE Name = 'tomcat7'"
    },{
      "name":"CPU",
      "query":"select * from Win32_PerfFormattedData_PerfOS_Processor"
    },{
      "name":"Logical Disk",
      "query":"select * from Win32_PerfFormattedData_PerfDisk_LogicalDisk"
    },{
      "name":"Physical Disk",
      "query":"select * from Win32_PerfFormattedData_PerfDisk_PhysicalDisk"
    },{
      "name":"Memory",
      "query":"select * from Win32_PerfFormattedData_PerfOS_Memory"
    },{
      "name":"Network",
      "query":"select * from Win32_PerfFormattedData_Tcpip_NetworkInterface"
    }],
    "sourceType":"LocalWindowsPerfMon",
    "alive":true
  },

Now, as I got hundreds of files like those, I wrote a foreach through the whole directory:

for filename in os.listdir('./json/'):
   with open('./json/'+filename) as data_file:    
   sources = json.load(data_file)

Now I would need something like again a foreach source in sources, which adds a row (or a entry or whatever a "line" in JSON is called) to every source (something like collectorName=fileName) and then overwrite the old file with the new one.

The JSON would then look like this:

   {
      "sources":[{
        "id":100012630,
        "name":"Activity Login Page",
        "category":"NAM/Activity",
        "automaticDateParsing":true,
        "multilineProcessingEnabled":false,
        "useAutolineMatching":false,
        "forceTimeZone":true,
        "timeZone":"Europe/Brussels",
        "filters":[],
        "cutoffTimestamp":1414364400000,
        "encoding":"UTF-8",
        "pathExpression":"C:\\NamLogs\\nam-login-page.log*",
        "blacklist":[],
        "sourceType":"LocalFile",
        "alive":true,
        "collectorName":"Collector2910"
      },{
        "id":100001824,
        "name":"localWinEvent",
        "category":"NAM/OS/EventLog",
        "automaticDateParsing":true,
        "multilineProcessingEnabled":false,
        "useAutolineMatching":false,
        "forceTimeZone":false,
        "filters":[],
        "cutoffTimestamp":1409090400000,
        "encoding":"UTF-8",
        "logNames":["Security","Application","System","Others"],
        "sourceType":"LocalWindowsEventLog",
        "alive":true,
        "collectorName":"Collector2910"
      },{.....

I hope I could explain my issue and I'd be happy if someone could help me out (even with a totally different solution).

Thanks in advance

Michael

chown · Accepted Answer · 2015-05-28 11:13:14Z

2

Here's one way to do it:

for filename in os.listdir('./json/'):
    sources = None
    with open('./json/'+filename) as data_file:    
        sources = json.load(data_file)
        sourcelist = sources['sources']
        for i, s in enumerate(sourcelist):
            sources['sources'][i]['collectorName'] = 'Collector' + str(i)
    with open('./json/'+filename, 'w') as data_file:  
        data_file.write(json.dumps(sources))

answered May 28, 2015 at 11:13

chown

52.9k17 gold badges138 silver badges170 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

bruno desthuilliers Over a year ago

why sources['sources'][i] when you have the record available as s ? And why data_file.write(json.dumps(sources)) when you have json.dump(obj, file) ?

chown Over a year ago

1) I prefer not to modify the list I'm iterating through. 2) They are equivalent, the json.dump methos does pretty much the same thing.

bruno desthuilliers Over a year ago

1. modifying a list you're iterating over is only dangerous when you add / remove items to / from that list, not when you're modifying the items already in the list - and in the current case, sources['sources'][i] is s == True - both names point to same object, really. 2. yes but the second is less code to write ;)

bruno desthuilliers · Accepted Answer · 2015-05-28 11:18:34Z

1

for filename in os.listdir('./json/'):
   with open('./json/'+filename) as data_file:    
       datadict = json.load(data_file)
   # At this point you have a plain python dict.
   # This dict has a 'sources' key, pointing to
   # a list of dicts. What you want is to add
   # a 'collectorName': filename key:value pair
   # to each of these dicts
   for record in datadict["sources"]:
       record["collectorName"] = filename
   # now you just have to serialize your datadict back
   # to json and write it back to the file - which is
   # in fact a single operation
   with open('./json/'+filename, "w") as data_file:    
       json.dump(datadict, data_file)

answered May 28, 2015 at 11:18

bruno desthuilliers

78.3k6 gold badges102 silver badges129 bronze badges

2 Comments

Michael Over a year ago

also in this try i run into: ValueError: No JSON object could be decoded

Michael Over a year ago

is it an issue with json.loads and json.load?

Collectives™ on Stack Overflow

Add "entry" to JSON File with Python

2 Answers 2

3 Comments

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

3 Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related