fixing up hostnames

This is the place for you, if you got rsyslog up and running but wonder how to make it do what you want.

Moderator: rgerhards

Google Ads


fixing up hostnames

Postby skunkwerks » Fri Nov 18, 2016 1:29 pm

set is as follows:

localhost -> rsyslogd -> [relp over spiped tunnel] -> rsyslog -> logfiles + splunk

1. fixing hostname=localhost

I have a few pesky daemons that log with hostname=localhost which is very confusing by the time it arrives at the central log server.

Given that my config mgmt system can write the correct hostname into rsyslog.conf, how best to overwrite the %hostname% variable?
I'm a bit overwhelmed after reading about templates etc.

2. sometimes hostname has fqdn and sometimes not

haproxy logs with the full hostname, but rsyslogd uses the short hostname, for example from imklog or imuxsock:

Code: Select all
2016-11-16T02:00:00.048314+00:00 bridget /usr/sbin/cron[14356]: (root) CMD (/usr/libexec/atrun)
2016-11-16T03:32:39+00:00 beatrix.us.example.com haproxy[20449] 123.45.67.89:39171 [16/Nov/2016:03:32:39.603] http couch/beatrix.us.example.com 12/0/0/2/14 200 340 - - ---- 0/0/0/0/0 0/0 "GET / HTTP/1.0"


How should I fix this? I can live with FQDN or short name either way.

thanks!

my config:

Code: Select all
# /usr/local/etc/rsyslog.conf
# Load Modules
module(load="imtcp")
module(load="imklog")
module(load="imudp")
module(load="imrelp")
module(load="omrelp")
module(load="imuxsock")
module(load="immark")
module(load="impstats")

$WorkDirectory /var/spool/rsyslog
$MaxMessageSize 64k
$MainMsgQueueFileName mainq
$MainMsgQueueType LinkedList
$MainMsgQueueSaveOnShutDown on
$MainMsgQueueMaxDiskSpace 10g
$MainMsgQueueSize 1m
$ActionSendResendLastMsgOnReconnect on
$ActionResumeRetryCount -1
$KLogPermitNonKernelFacility on

# rsyslog templates

# rsyslog input modules
input(type="imtcp"
    address="127.0.0.1"
    port="514")
input(type="imudp"
    address="127.0.0.1"
    port="514")

# filters

:msg, contains, "NSSWITCH(_nsdispatch)"  ~

# rulesets

# rsyslog output modules

# base servers forward logs
action(type="omrelp"
    target="127.0.0.1"
    port="44514"
)

# log locally just like normal syslog
action(type="omfile"
    queue.type="linkedlist"
        name="rsyslog"
        file="/var/log/messages"
)
skunkwerks
New
 
Posts: 4
Joined: Fri Nov 18, 2016 1:04 pm

Urgent Question?

  • Pulling out your Hair?
  • Wasting Time and Money?
  • Deadline Approaching?

Re: fixing up hostnames

Postby dlang » Fri Nov 18, 2016 8:11 pm

you will need to create a template that uses %$.myhostname% instead of %hostname% and then

set $.myhostname = $hostname;

followed by the special cases.

take a look at the values of $fromhost and $fromhost-ip for your problem machines (you can either make a template that includes them for testing, or log with RSYSLOG_DebugFormat)

RFC3184 specified that hostnames should be the short hostname, RFC5424 specifies that hostnames should be FQDN

by default, rsyslog will send the short hostname and convert a fqdn to a short name when processing (all configurable)
dlang
Frequent Poster
 
Posts: 1001
Joined: Mon Sep 15, 2008 7:44 am

Re: fixing up hostnames

Postby skunkwerks » Fri Nov 18, 2016 8:41 pm

Here's what I came up with so far, however this matches on *any* text and not necessarily the %hostname% field. How could I do that?

Code: Select all
if re_match($msg,'(localhost)')
then
{
    set $!ext = re_extract($msg,'(localhost',0,1,"");
    set $!msg= replace($msg, $!ext, "wintermute");
}
else
    set $!msg = $msg;
skunkwerks
New
 
Posts: 4
Joined: Fri Nov 18, 2016 1:04 pm

Re: fixing up hostnames

Postby skunkwerks » Fri Nov 18, 2016 8:42 pm

Oops didn't see your reply, I'll look more into the template. Thanks!
skunkwerks
New
 
Posts: 4
Joined: Fri Nov 18, 2016 1:04 pm

Re: fixing up hostnames

Postby PCnetMD » Thu Jul 13, 2017 5:12 pm

Did this get resolved?
If so, can you share what you did?
Thank you.
PCnetMD
New
 
Posts: 3
Joined: Fri Jun 30, 2017 2:14 pm

Re: fixing up hostnames

Postby skunkwerks » Tue Sep 26, 2017 3:59 pm

yes, along these lines:

Code: Select all
template(name="normalised" type="string"
    string="<%pri%>%protocol-version% %timestamp:::date-rfc3339% {{ inventory_hostname_short }} %syslogtag%%msg:::sp-if-no-1st-sp%%msg:::drop-last-lf%\n")

action(type="omrelp"
    target="127.0.0.1"
    template="normalised"
    ...
)


I should probably use a standard RFC5424 template but this has proved sufficient in my specific environment.
skunkwerks
New
 
Posts: 4
Joined: Fri Nov 18, 2016 1:04 pm

Google Ads



Return to Configuration

Who is online

Users browsing this forum: No registered users and 0 guests