Currently we have the following config for syslog tdagent(fluentd) config and would like to create another field for priority:Error for my log. How can I do this?
Log:
Nov 11 00:18:57 Build01v nagios: SERVICE ALERT: mmj21;Dropwizard MMJ Thread Pool;UNKNOWN;SOFT;1;**Error**: unable to access dropwizard metrics at localhost using port 8001
Current config:
<source>
@type tail
path /var/log/messages
pos_file /var/log/td-agent/var_log_messages.pos
read_from_head true
tag /var/log/messages
<parse>
@type regexp
expression ^(?<time>[^ ]* [^ ]* [^ ]*) (?<host>[^ ]*) (?<process>[^ ]*): (?<message>.*)$
time_format %b %d %H:%M:%S
time_key time
</parse>
</source>
Output:
https://fluentular.herokuapp.com/parse?regexp=%5E%28%3F%3Ctime%3E%5B%5E+%5D*+%5B%5E+%5D*+%5B%5E+%5D*%29+%28%3F%3Chost%3E%5B%5E+%5D*%29+%28%3F%3Cprocess%3E%5B%5E+%5D*%29%3A+%28%3F%3Cmessage%3E.*%29%24&input=Nov+11+00%3A18%3A57+Build01v+nagios%3A+SERVICE+ALERT%3A+mmj21%3BDropwizard+MMJ+Thread+Pool%3BUNKNOWN%3BSOFT%3B1%3B**Error%3A**+unable+to+access+dropwizard+metrics+at+localhost+using+port+8001&time_format=%25b+%25d+%25H%3A%25M%3A%25S
Records
Key Value
host Build01v
process nagios
message SERVICE ALERT: mmj21;Dropwizard MMJ Thread Pool;UNKNOWN;SOFT;1;**Error:** unable to access dropwizard metrics at localhost using port 8001
I'm guessing that maybe,
^(?<time>\S* \S* \S*) (?<host>\S*) (?<process>[^:]*): (?<priority>[^:]*):(?<message>.*)$
might be what you're trying to write.
If you wish to simplify/modify/explore the expression, it's been explained on the top right panel of regex101.com. If you'd like, you can also watch in this link, how it would match against some sample inputs.
jex.im visualizes regular expressions: