0

I am getting syslog data from my generator, and I would like to extract the date and time at the beginning of the line and the action that is at the end of the file.

what I have so far is simply this:

awk -F ' ' '{print $1,$2,$3,":",$NF}' test.log

my output has incomplete log messages

Sep 17 20:47:02 : Warning
Sep 20 12:02:20 : Started
Sep 20 12:09:31 : Level
Sep 20 12:10:27 : Level
Sep 20 12:10:28 : Stopped
Sep 23 09:19:03 : Auto
Sep 23 09:19:04 : Cleared
Sep 23 09:19:13 : Auto
Sep 30 12:33:48 : Auto
Sep 30 12:33:55 : Started
Sep 30 12:41:14 : Level
Sep 30 12:41:56 : Stopped
Sep 30 12:41:58 : Cleared
Oct 3 14:00:52 : Started
Oct 3 14:15:17 : Stopped
Oct 3 14:20:39 : Auto
Oct 15 12:02:55 : Started
Oct 15 12:12:50 : Stopped

This is what I'm trying to print out; the actual date and time and the full action message:

Date Time Fuel has High Warning
Date Time Genset Started
Date Time  Engine Coolant Temperature - Data Valid but Above Normal Operational Range - Moderately Severe Level
Date Time  Engine Coolant Temperature - Data Valid but Above Normal Operational Range - Most Severe Level
Date Time  Genset Stopped
Date Time  Genset Not In Auto
Date Time  All Faults Cleared
Date Time  Genset in Auto
Date Time  Genset Not In Auto
Date Time  Genset Started
Date Time  Engine Coolant Temperature - Data Valid but Above Normal Operational Range - Moderately Severe Level
Date Time  Genset Stopped
Date Time  All Faults Cleared
Date Time  Genset Started
Date Time  Genset Stopped
Date Time  Genset in Auto
Date Time  Genset Started
Date Time  Genset Stopped

The length of each line has no set length.

this is test file

Sep 17 20:47:02 192.168.22.60 0��#002#001#001#004#005AE4ML���#002#001#011#002#001#000#002#001#0000��0#015#006#010+#006#001#002#001#001#003#000C#001#0000#030#006#012+#006#001#006#003#001#001#004#001#000#006#012+#006#001#004#001��s#001#0000##006#013+#006#001#004#001��s#001#001#000#004#0249/17/2023 8:47:01 PM0#023#006#013+#006#001#004#001��s#001#002#000#004#004Fuel0#023#006#013+#006#001#004#001��s#001#004#000#004#00445940#026#006#013+#006#001#004#001��s#001#003#000#004#007Warning0$#006#013+#006#001#004#001��s#001#005#000#004#025Fuel has High Warning
Sep 20 12:02:20 192.168.22.60 0��#002#001#001#004#005AE4ML���#002#001#012#002#001#000#002#001#0000��0#015#006#010+#006#001#002#001#001#003#000C#001#0000#030#006#012+#006#001#006#003#001#001#004#001#000#006#012+#006#001#004#001��s#001#0000$#006#013+#006#001#004#001��s#001#001#000#004#0259/20/2023 12:02:23 PM0#033#006#013+#006#001#004#001��s#001#002#000#004#01412615_lussie0#023#006#013+#006#001#004#001��s#001#004#000#004#00451430#032#006#013+#006#001#004#001��s#001#003#000#004#013Information0#035#006#013+#006#001#004#001��s#001#005#000#004#016Genset Started
Sep 20 12:09:31 192.168.22.60 0�#001(#002#001#001#004#005AE4ML��#001#032#002#001#013#002#001#000#002#001#0000�#001#0150#015#006#010+#006#001#002#001#001#003#000C#001#0000#030#006#012+#006#001#006#003#001#001#004#001#000#006#012+#006#001#004#001��s#001#0000$#006#013+#006#001#004#001��s#001#001#000#004#0259/20/2023 12:09:34 PM0#033#006#013+#006#001#004#001��s#001#002#000#004#01412615_lussie0#022#006#013+#006#001#004#001��s#001#004#000#004#0031460#026#006#013+#006#001#004#001��s#001#003#000#004#007Warning0s#006#013+#006#001#004#001��s#001#005#000#004dEngine Coolant Temperature - Data Valid but Above Normal Operational Range - Moderately Severe Level
Sep 20 12:10:27 192.168.22.60 0�#001##002#001#001#004#005AE4ML��#001#025#002#001#014#002#001#000#002#001#0000�#001#0100#015#006#010+#006#001#002#001#001#003#000C#001#0000#030#006#012+#006#001#006#003#001#001#004#001#000#006#012+#006#001#004#001��s#001#0000$#006#013+#006#001#004#001��s#001#001#000#004#0259/20/2023 12:10:30 PM0#033#006#013+#006#001#004#001��s#001#002#000#004#01412615_lussie0#022#006#013+#006#001#004#001��s#001#004#000#004#0031510#027#006#013+#006#001#004#001��s#001#003#000#004#010Shutdown0m#006#013+#006#001#004#001��s#001#005#000#004^Engine Coolant Temperature - Data Valid but Above Normal Operational Range - Most Severe Level
Sep 20 12:10:28 192.168.22.60 0��#002#001#001#004#005AE4ML���#002#001#015#002#001#000#002#001#0000��0#015#006#010+#006#001#002#001#001#003#000C#001#0000#030#006#012+#006#001#006#003#001#001#004#001#000#006#012+#006#001#004#001��s#001#0000$#006#013+#006#001#004#001��s#001#001#000#004#0259/20/2023 12:10:30 PM0#033#006#013+#006#001#004#001��s#001#002#000#004#01412615_lussie0#023#006#013+#006#001#004#001��s#001#004#000#004#00451440#032#006#013+#006#001#004#001��s#001#003#000#004#013Information0#035#006#013+#006#001#004#001��s#001#005#000#004#016Genset Stopped
Sep 23 09:19:03 192.168.22.60 0��#002#001#001#004#005AE4ML���#002#001#016#002#001#000#002#001#0000��0#015#006#010+#006#001#002#001#001#003#000C#001#0000#030#006#012+#006#001#006#003#001#001#004#001#000#006#012+#006#001#004#001��s#001#0000##006#013+#006#001#004#001��s#001#001#000#004#0249/23/2023 9:19:05 AM0#033#006#013+#006#001#004#001��s#001#002#000#004#01412615_lussie0#023#006#013+#006#001#004#001��s#001#004#000#004#00451380#032#006#013+#006#001#004#001��s#001#003#000#004#013Information0!#006#013+#006#001#004#001��s#001#005#000#004#022Genset Not In Auto
Sep 23 09:19:04 192.168.22.60 0��#002#001#001#004#005AE4ML���#002#001#017#002#001#000#002#001#0000��0#015#006#010+#006#001#002#001#001#003#000C#001#0000#030#006#012+#006#001#006#003#001#001#004#001#000#006#012+#006#001#004#001��s#001#0000##006#013+#006#001#004#001��s#001#001#000#004#0249/23/2023 9:19:07 AM0#033#006#013+#006#001#004#001��s#001#002#000#004#01412615_lussie0#023#006#013+#006#001#004#001��s#001#004#000#004#00472190#032#006#013+#006#001#004#001��s#001#003#000#004#013Information0!#006#013+#006#001#004#001��s#001#005#000#004#022All Faults Cleared
Sep 23 09:19:13 192.168.22.60 0��#002#001#001#004#005AE4ML���#002#001#020#002#001#000#002#001#0000��0#015#006#010+#006#001#002#001#001#003#000C#001#0000#030#006#012+#006#001#006#003#001#001#004#001#000#006#012+#006#001#004#001��s#001#0000##006#013+#006#001#004#001��s#001#001#000#004#0249/23/2023 9:19:16 AM0#033#006#013+#006#001#004#001��s#001#002#000#004#01412615_lussie0#023#006#013+#006#001#004#001��s#001#004#000#004#00456640#032#006#013+#006#001#004#001��s#001#003#000#004#013Information0#035#006#013+#006#001#004#001��s#001#005#000#004#016Genset in Auto
Sep 30 12:33:48 192.168.22.60 0��#002#001#001#004#005AE4ML���#002#001#021#002#001#000#002#001#0000��0#015#006#010+#006#001#002#001#001#003#000C#001#0000#030#006#012+#006#001#006#003#001#001#004#001#000#006#012+#006#001#004#001��s#001#0000$#006#013+#006#001#004#001��s#001#001#000#004#0259/30/2023 12:33:50 PM0#033#006#013+#006#001#004#001��s#001#002#000#004#01412615_lussie0#023#006#013+#006#001#004#001��s#001#004#000#004#00451380#032#006#013+#006#001#004#001��s#001#003#000#004#013Information0!#006#013+#006#001#004#001��s#001#005#000#004#022Genset Not In Auto
Sep 30 12:33:55 192.168.22.60 0��#002#001#001#004#005AE4ML���#002#001#022#002#001#000#002#001#0000��0#015#006#010+#006#001#002#001#001#003#000C#001#0000#030#006#012+#006#001#006#003#001#001#004#001#000#006#012+#006#001#004#001��s#001#0000$#006#013+#006#001#004#001��s#001#001#000#004#0259/30/2023 12:33:57 PM0#033#006#013+#006#001#004#001��s#001#002#000#004#01412615_lussie0#023#006#013+#006#001#004#001��s#001#004#000#004#00451430#032#006#013+#006#001#004#001��s#001#003#000#004#013Information0#035#006#013+#006#001#004#001��s#001#005#000#004#016Genset Started
Sep 30 12:41:14 192.168.22.60 0�#001(#002#001#001#004#005AE4ML��#001#032#002#001#023#002#001#000#002#001#0000�#001#0150#015#006#010+#006#001#002#001#001#003#000C#001#0000#030#006#012+#006#001#006#003#001#001#004#001#000#006#012+#006#001#004#001��s#001#0000$#006#013+#006#001#004#001��s#001#001#000#004#0259/30/2023 12:41:17 PM0#033#006#013+#006#001#004#001��s#001#002#000#004#01412615_lussie0#022#006#013+#006#001#004#001��s#001#004#000#004#0031460#026#006#013+#006#001#004#001��s#001#003#000#004#007Warning0s#006#013+#006#001#004#001��s#001#005#000#004dEngine Coolant Temperature - Data Valid but Above Normal Operational Range - Moderately Severe Level
Sep 30 12:41:56 192.168.22.60 0��#002#001#001#004#005AE4ML���#002#001#024#002#001#000#002#001#0000��0#015#006#010+#006#001#002#001#001#003#000C#001#0000#030#006#012+#006#001#006#003#001#001#004#001#000#006#012+#006#001#004#001��s#001#0000$#006#013+#006#001#004#001��s#001#001#000#004#0259/30/2023 12:41:59 PM0#033#006#013+#006#001#004#001��s#001#002#000#004#01412615_lussie0#023#006#013+#006#001#004#001��s#001#004#000#004#00451440#032#006#013+#006#001#004#001��s#001#003#000#004#013Information0#035#006#013+#006#001#004#001��s#001#005#000#004#016Genset Stopped
Sep 30 12:41:58 192.168.22.60 0��#002#001#001#004#005AE4ML���#002#001#025#002#001#000#002#001#0000��0#015#006#010+#006#001#002#001#001#003#000C#001#0000#030#006#012+#006#001#006#003#001#001#004#001#000#006#012+#006#001#004#001��s#001#0000$#006#013+#006#001#004#001��s#001#001#000#004#0259/30/2023 12:42:01 PM0#033#006#013+#006#001#004#001��s#001#002#000#004#01412615_lussie0#023#006#013+#006#001#004#001��s#001#004#000#004#00472190#032#006#013+#006#001#004#001��s#001#003#000#004#013Information0!#006#013+#006#001#004#001��s#001#005#000#004#022All Faults Cleared
Oct  3 14:00:52 192.168.22.60 0��#002#001#001#004#005AE4ML���#002#001#026#002#001#000#002#001#0000��0#015#006#010+#006#001#002#001#001#003#000C#001#0000#030#006#012+#006#001#006#003#001#001#004#001#000#006#012+#006#001#004#001��s#001#0000##006#013+#006#001#004#001��s#001#001#000#004#02410/3/2023 2:00:55 PM0#033#006#013+#006#001#004#001��s#001#002#000#004#01412615_lussie0#023#006#013+#006#001#004#001��s#001#004#000#004#00451430#032#006#013+#006#001#004#001��s#001#003#000#004#013Information0#035#006#013+#006#001#004#001��s#001#005#000#004#016Genset Started
Oct  3 14:15:17 192.168.22.60 0��#002#001#001#004#005AE4ML���#002#001#027#002#001#000#002#001#0000��0#015#006#010+#006#001#002#001#001#003#000C#001#0000#030#006#012+#006#001#006#003#001#001#004#001#000#006#012+#006#001#004#001��s#001#0000##006#013+#006#001#004#001��s#001#001#000#004#02410/3/2023 2:15:20 PM0#033#006#013+#006#001#004#001��s#001#002#000#004#01412615_lussie0#023#006#013+#006#001#004#001��s#001#004#000#004#00451440#032#006#013+#006#001#004#001��s#001#003#000#004#013Information0#035#006#013+#006#001#004#001��s#001#005#000#004#016Genset Stopped
Oct  3 14:20:39 192.168.22.60 0��#002#001#001#004#005AE4ML���#002#001#030#002#001#000#002#001#0000��0#015#006#010+#006#001#002#001#001#003#000C#001#0000#030#006#012+#006#001#006#003#001#001#004#001#000#006#012+#006#001#004#001��s#001#0000##006#013+#006#001#004#001��s#001#001#000#004#02410/3/2023 2:20:42 PM0#033#006#013+#006#001#004#001��s#001#002#000#004#01412615_lussie0#023#006#013+#006#001#004#001��s#001#004#000#004#00456640#032#006#013+#006#001#004#001��s#001#003#000#004#013Information0#035#006#013+#006#001#004#001��s#001#005#000#004#016Genset in Auto
Oct 15 12:02:55 192.168.22.60 0��#002#001#001#004#005AE4ML���#002#001#031#002#001#000#002#001#0000��0#015#006#010+#006#001#002#001#001#003#000C#001#0000#030#006#012+#006#001#006#003#001#001#004#001#000#006#012+#006#001#004#001��s#001#0000%#006#013+#006#001#004#001��s#001#001#000#004#02610/15/2023 12:02:59 PM0#033#006#013+#006#001#004#001��s#001#002#000#004#01412615_lussie0#023#006#013+#006#001#004#001��s#001#004#000#004#00451430#032#006#013+#006#001#004#001��s#001#003#000#004#013Information0#035#006#013+#006#001#004#001��s#001#005#000#004#016Genset Started
Oct 15 12:12:50 192.168.22.60 0��#002#001#001#004#005AE4ML���#002#001#032#002#001#000#002#001#0000��0#015#006#010+#006#001#002#001#001#003#000C#001#0000#030#006#012+#006#001#006#003#001#001#004#001#000#006#012+#006#001#004#001��s#001#0000%#006#013+#006#001#004#001��s#001#001#000#004#02610/15/2023 12:12:54 PM0#033#006#013+#006#001#004#001��s#001#002#000#004#01412615_lussie0#023#006#013+#006#001#004#001��s#001#004#000#004#00451440#032#006#013+#006#001#004#001��s#001#003#000#004#013Information0#035#006#013+#006#001#004#001��s#001#005#000#004#016Genset Stopped
Improve this question
9
  • 1
    For starters it looks like you may be using the wrong character set encoding for the data. (This could explain the �� characters in the data.) Commented Oct 19, 2023 at 23:03
  • Please edit your question and add the output you expect from the input file you show. Your awk is designed to produce exactly what you show, so if that isn't what you need, you need to tell us what is. Commented Oct 19, 2023 at 23:03
  • The output I'm looking for is this. Commented Oct 19, 2023 at 23:08
  • Not in the comments. Put updates in your question. Formatted like the other parts have been formatted. Commented Oct 19, 2023 at 23:12
  • Do you really want Date Time at the beginning of each line? Commented Oct 19, 2023 at 23:17

2 Answers 2

Reset to default
1

It does look a little like you may be using the wrong character set encoding for the data. (This could explain the �� characters in the data.) You should investigate the reference documentation for the output you're processing to see how it should be interpreted.

However, given it's "mostly right", it looks like the field delimiter is mostly # so you can start with that:

awk -F'#' '{ print $1, $NF }'

Sample output

Sep 17 20:47:02 192.168.22.60 0�� 025Fuel has High Warning
Sep 20 12:02:20 192.168.22.60 0�� 016Genset Started
Sep 20 12:09:31 192.168.22.60 0� 004dEngine Coolant Temperature - Data Valid but Above Normal Operational Range - Moderately Severe Level
Sep 20 12:10:27 192.168.22.60 0� 004^Engine Coolant Temperature - Data Valid but Above Normal Operational Range - Most Severe Level

Now we need to split the first set of space-separated fields, and strip what empirically appears to be three characters from the beginning of the "last" field:

awk -F'#' '{ split($1, a, " "); print a[1], a[2], a[3], substr($NF,4,length($NF)) }'

Sample output

Sep 17 20:47:02 Fuel has High Warning
Sep 20 12:02:20 Genset Started
Sep 20 12:09:31 dEngine Coolant Temperature - Data Valid but Above Normal Operational Range - Moderately Severe Level
Sep 20 12:10:27 ^Engine Coolant Temperature - Data Valid but Above Normal Operational Range - Most Severe Level

There's nothing obvious to show how or why the leading d and ^ should be removed from the third and fourth lines so they remain.

Improve this answer
0
0

Using any POSIX awk:

$ awk '{sub(/[0-9]+\..*(#[0-9]{3}){4}[^[:upper:]]*/,"")} 1' file
Sep 17 20:47:02 Fuel has High Warning
Sep 20 12:02:20 Genset Started
Sep 20 12:09:31 Engine Coolant Temperature - Data Valid but Above Normal Operational Range - Moderately Severe Level
Sep 20 12:10:27 Engine Coolant Temperature - Data Valid but Above Normal Operational Range - Most Severe Level
Sep 20 12:10:28 Genset Stopped
Sep 23 09:19:03 Genset Not In Auto
Sep 23 09:19:04 All Faults Cleared
Sep 23 09:19:13 Genset in Auto
Sep 30 12:33:48 Genset Not In Auto
Sep 30 12:33:55 Genset Started
Sep 30 12:41:14 Engine Coolant Temperature - Data Valid but Above Normal Operational Range - Moderately Severe Level
Sep 30 12:41:56 Genset Stopped
Sep 30 12:41:58 All Faults Cleared
Oct  3 14:00:52 Genset Started
Oct  3 14:15:17 Genset Stopped
Oct  3 14:20:39 Genset in Auto
Oct 15 12:02:55 Genset Started
Oct 15 12:12:50 Genset Stopped

The above assumes you always have a series of at least 4 #NNN strings at the end of the part you want removed and that you won't have 4 #NNN strings in the remaining part you want to print but it could contain #s in other contexts.

It also assumes your action part to be printed always starts with an upper case letter because otherwise I don't see any way to know we need to remove d from the start of dEngine ... on line 3 of the input as you want according to the expected output.

You can do the same with sed, e.g. using a sed that supports -E for EREs such as GNU or BSD sed:

$ sed -E 's/[0-9]+\..*(#[0-9]{3}){4}[^[:upper:]]*//' file
Sep 17 20:47:02 Fuel has High Warning
Sep 20 12:02:20 Genset Started
Sep 20 12:09:31 Engine Coolant Temperature - Data Valid but Above Normal Operational Range - Moderately Severe Level
Sep 20 12:10:27 Engine Coolant Temperature - Data Valid but Above Normal Operational Range - Most Severe Level
Sep 20 12:10:28 Genset Stopped
Sep 23 09:19:03 Genset Not In Auto
Sep 23 09:19:04 All Faults Cleared
Sep 23 09:19:13 Genset in Auto
Sep 30 12:33:48 Genset Not In Auto
Sep 30 12:33:55 Genset Started
Sep 30 12:41:14 Engine Coolant Temperature - Data Valid but Above Normal Operational Range - Moderately Severe Level
Sep 30 12:41:56 Genset Stopped
Sep 30 12:41:58 All Faults Cleared
Oct  3 14:00:52 Genset Started
Oct  3 14:15:17 Genset Stopped
Oct  3 14:20:39 Genset in Auto
Oct 15 12:02:55 Genset Started
Oct 15 12:12:50 Genset Stopped
Improve this answer

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.