Parse a log file into json object

Question

I have log file named request.log this is the content of the log file

[2022-06-30T09:56:40.146Z] ### POST https://test.csdf/auth/send_otp  
    
 {
  "method": "POST",
  "headers": {
    "User-Agent": "testing",
    "Content-Type": "application/json"
  }
}
[2022-06-30T09:56:40.668Z] ### POST https://test.csdf/auth/login  
    
 {
  "method": "POST",
  "headers": {
    "User-Agent": "testing",
    "Content-Type": "application/json"
  }
}

how can I parse the log file and transform it into array json object like this

[
  {
     "method": "POST",
     "url": "https://test.co/otp"
  },
  {
     "method": "POST",
     "url": "https://test.co/login"
  }
]

Neither https://test.co/otp nor https://test.co/login are actually anywhere to be found in your sample data. So do you actually want to manipulate those URLs from the log file ... or just pay a little more attention when asking? — C3roe
– C3roe, Commented Jun 30, 2022 at 13:26
If you want the URLs as they are - then just split this into individual lines and loop over them; check if a line starts with [, and if so, extract method & URL, if not, skip it and process the next one ... — C3roe
– C3roe, Commented Jun 30, 2022 at 13:28
@NS23 I actually have this http request function, and I put fs.createWriteStream() in there to log the request payload. — N Hilmi
– N Hilmi, Commented Jul 1, 2022 at 2:53

Haris Bouchlis · Accepted Answer · 2022-06-30 13:21:13Z

2

You have to run some kind of script to convert them to JSON. In javascript/nodejs you could do something like this:

const logFileContents = `[2022-06-30T09:56:40.146Z] ### POST https://test.csdf/auth/send_otp  
    
{
 "method": "POST",
 "headers": {
   "User-Agent": "testing",
   "Content-Type": "application/json"
 }
}
[2022-06-30T09:56:40.668Z] ### POST https://test.csdf/auth/login  
   
{
 "method": "POST",
 "headers": {
   "User-Agent": "testing",
   "Content-Type": "application/json"
 }
}`;

function convertToJSON(logs) {
  return logs
    .match(/\[.+\]\s\#+\s\w+\s.*\n/g) // fetch lines to parse
    .map((line) => line.split('###')[1].trim().split(' ')) //isolate method/url
    .map((log) => ({
      method: log[0],
      url: log[1],
    }));// convert array to object
}

console.log(JSON.stringify(convertToJSON(logFileContents), null, 4));

if it doesn't work for your log files, just fix the regex to match your case

edited Jun 30, 2022 at 13:21

answered Jun 30, 2022 at 13:05

Haris Bouchlis

2,5861 gold badge25 silver badges39 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

N Hilmi Over a year ago

this is what I'm looking for. I'm struggling on how to split it and map it into a new JSON. Thankyou!

Collectives™ on Stack Overflow

Parse a log file into json object

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related