MonitorWare Knowledge Base

If you have a moment, would you answer a quick question for me. I have rsyslog setup on a RedHat linux server running with UDP Receive enabled. I am forwarding messages to this server from both RedHat ES (versions 3 and 4) as well as AIX servers running 5.2 or higher. I then have these feeding a MySQL database. The one weird thing that I am getting is that the FromHost field becomes Message instead of the name of the machine or the IP address of the machine. I am running msyslog on another server and I do not have this error. Is this something I am doing?

This is an example of a relayed message from the AIX machine:

Mar 4 19:28:46 140.90.85.147 Message forwarded from tgp16: syslogd: restart

Thanks for the great product and I am looking forward to anything and everything new.

• Custom written rsyslog.conf?
• Maintenance Contract?
• Installation support?

Larry,

I guess this is a somewhat older build. Please do an "rsyslogd -v" and check that it is the most recent one. If so, please post me some debug information. You can enable that by running rsyslogd interactively, that is by adding the "-n -d" command line options. It will spit out diagnostics to stdout. Let me know roughly 100 lines around a message reception from AIX.

Rainer

Confusion hostname!

When AIX send message log to rsyslog server on RedHat, rsyslog received wrong hostname. Hostname which rsyslog receive is Message (true is t24_db02).
I debug the following command : rsyslogd -r 514 -m 0 -d -n
Output of command above:

Successful select, descriptor count = 1, Activity on: 5
-1208068416: Message from UDP inetd socket: #5, host: 192.168.200.153
-1208068416: Message length: 184, File descriptor: 5.
-1208068416: logmsg: daemon.debug<31>, flags 2, from 't24_db02', msg Dec 1 14:39:40 Message forwarded from t24_db02: nrpe[257628]: Command completed with return code 0 and output: DISK OK - free space: /opt 91 MB (18%);| /opt=421MB;460;506;0;512
-1208068416: Message has legacy syslog format.
-1208068416: EnqueueMsg signaled condition (0)
-1208068416: Listening on syslog UDP port.
-1208068416: ----------------------------------------
Calling select, active file descriptors (max 5): 3 5
-1208068416:
Successful select, descriptor count = 1, Activity on: 5
-1208068416: Message from UDP inetd socket: #5, host: 192.168.200.153
-1208068416: Message length: 158, File descriptor: 5.
-1208068416: logmsg: daemon.debug<31>, flags 2, from 't24_db02', msg Dec 1 14:39:40 Message forwarded from t24_db02: nrpe[257628]: Return Code: 0, Output: DISK OK - free space: /opt 91 MB (18%);| /opt=421MB;460;506;0;512
-1208068416: Message has legacy syslog format.
-1208068416: EnqueueMsg signaled condition (0)
-1208068416: Listening on syslog UDP port.
-1208068416: ----------------------------------------
Calling select, active file descriptors (max 5): 3 5
-1208068416:
Successful select, descriptor count = 1, Activity on: 5
-1208068416: Message from UDP inetd socket: #5, host: 192.168.200.153
-1208068416: Message length: 104, File descriptor: 5.
-1208068416: logmsg: daemon.debug<31>, flags 2, from 't24_db02', msg Dec 1 14:39:40 Message forwarded from t24_db02: nrpe[257628]: Connection from 10.1.10.100 closed.
-1208068416: Message has legacy syslog format.
-1208068416: EnqueueMsg signaled condition (0)
-1208068416: Listening on syslog UDP port.
-1208068416: ----------------------------------------
Calling select, active file descriptors (max 5): 3 5
-1208071248: Lone worker is running...
-1208071248: Called fprintlog, logging to MYSQL
-1208071248: Lone worker is running...
-1208071248: Called fprintlog, logging to MYSQL
-1208071248: Lone worker is running...
-1208071248: Called fprintlog, logging to MYSQL
-1208071248: Lone worker is running...
-1208071248: Called fprintlog, logging to MYSQL
-1208071248: Lone worker is running...
-1208071248: Called fprintlog, logging to MYSQL
-1208071248: Lone worker is running...
-1208071248: Called fprintlog, logging to MYSQL
-1208071248: singleWorker: queue EMPTY, waiting for next message.
-1208068416: Select interrupted.
-1208068416: Listening on syslog UDP port.
-1208068416: ----------------------------------------

I try both of rsyslog version rsyslogd 1.12.2 and rsyslogd 1.12.3. Result is same.
Please help me!

From what I can see, the hostname is correctly received. Please post your rsyslog.conf, so that I can have a look at it. It would also be great, if you could provide me with some output file lines from a wrong message.

Rainer

The following file :
rsyslog server on Redhat Linux;
/etc/hosts
# Do not remove the following line, or various programs
# that require network functionality will fail.
127.0.0.1 localhost.localdomain localhost
10.1.9.76 linux
192.168.200.20 t24_as02
192.168.200.153 t24_db02

/etc/rsyslog.conf
# Below find some samples of what a template can do. Have a good
# time finding out what they do [or just tun them]

# A template that resambles traditional syslogd file output:
$template TraditionalFormat,"%timegenerated% %HOSTNAME% %syslogtag%%msg:::drop-last-lf%\n"

# A template that tells you a little more about the message:
$template precise,"%syslogpriority%,%syslogfacility%,%timegenerated:::fulltime%,%HOSTNAME%,%syslogtag%,%msg%\n"

# A template that resembles RFC 3164 on-the-wire format:
# (yes, there is NO space betwen syslogtag and msg! that's important!)
$template RFC3164fmt,"<%PRI%>%TIMESTAMP% %HOSTNAME% %syslogtag%%msg%"

# a template resembling traditional wallmessage format:
$template wallmsg,"\r\n\7Message from syslogd@%HOSTNAME% at %timegenerated% ...\r\n %syslogtag%%msg%\n\r"

# The template below emulates winsyslog format, but we need to check the time
# stamps used. for now, it is good enough This format works best with
# other members of the MonitorWare product family. It is also a good sample
# where you can see the property replacer in action.
$template WinSyslogFmt,"%HOSTNAME%,%timegenerated:1:10:date-rfc3339%,%timegenerated:12:19:date-rfc3339%,%timegenerated:1:10:date-rfc3339%,%timegenerated:12:19:date-rfc3339%,%syslogfacility%,%syslogpriority%,%syslogtag%%msg%\n"


# A template used for database writing (notice it *is* an actual
# sql-statement):
$template dbFormat,"insert into SystemEvents (Message, Facility,FromHost, Priority, DeviceReportedTime, ReceivedAt, InfoUnitID, SysLogTag) values ('%msg%', %syslogfacility%, '%HOSTNAME%',%syslogpriority%, '%timereported:::date-mysql%', '%timegenerated:::date-mysql%', %iut%, '%syslogtag%')",sql

# Selector lines are somewhat different from stock syslogd. With
# rsyslog, you can add a semicolon ";" after the target and then
# the template name. That will assign this template to the respective
# action. If no template name is given, a hardcoded template is used.
# If a template name is given, but the template was not defined, the
# selector line is DEACTIVATED.
#
# #############
# # IMPORTANT #
# #############

# #############
# # IMPORTANT #
# #############
# Templates MUST be defined BEFORE they are used! It is OK to
# intermix template definitions and selector lines within the
# config file, but each template MUST be defined before it is
# used the first time!
#

# We have some very rough samples here - This sample file focusses
# on the new syntax, so we do NOT describe all possible selections.
# Use the syslog.conf if you are interested to see how to select
# based on facility and severits (aka priority).


# And this finally is a database action
# The semicolon at the end is theoreticaly not necessary,
# but the current rsyslogd has a bug that makes it abort if
# it is missing. As Dennis Olvany pointed out, it would e
# extremely nice to have this semicolon in the sample conf
*.* >localhost,Syslog,root,123456;dbFormat
# It uses the default schema (MonitorWare format). The parameters
# should be self-explanatory.


ON AIX Server (192.168.200.153)
/etc/hosts

127.0.0.1 loopback localhost
192.168.200.153 t24_db02
10.1.9.76 linux

/etc/syslog.conf
*.debug @linux

Output PHPlogCon
Event 1 to 20 from 11914 «« « 1 2 3 4 » »»
Date Facility Severity InfoUnit Host Message
16:29:32 3 DEBUG SL Message from t24_db02: nrpe[17674]: Host is asking for command 'check_disk/opt' to be run...
16:29:32 3 DEBUG SL Message from t24_db02: nrpe[17674]: Running command: /usr/local/nagios/nagios/libexec/check_disk -w 10% -c 1% -p /dev/hd10opt
16:29:32 3 DEBUG SL Message from t24_db02: nrpe[17674]: Command completed with return code 0 and output: DISK OK - free space: /opt 91 MB (18%);| /opt=421MB;460;506;0;512
16:29:32 3 DEBUG SL Message from t24_db02: nrpe[17674]: Return Code: 0, Output: DISK OK - free space: /opt 91 MB (18%);| /opt=421MB;460;506;0;512
16:29:32 3 DEBUG SL Message from t24_db02: nrpe[17674]: Connection from 10.1.10.100 closed.
16:29:31 3 DEBUG SL Message from t24_db02: nrpe[17674]: Connection from 10.1.10.100 port 51576
16:29:31 3 DEBUG SL Message from t24_db02: nrpe[17674]: Host address is in allowed_hosts
16:29:31 3 DEBUG SL Message from t24_db02: nrpe[17674]: Handling the connection...
16:29:24 3 DEBUG SL Message from t24_db02: nrpe[39792]: Host is asking for command 'check_disk/globus' to be run...



Please help me!

ahh... I begin to understand. I looks like AIX has a very strange message format. I need to have a peek at the raw format of the message, but I do not have the necessary code at hand. Will post that later. Or you can find out yourself: the property is "%rawmsg%" - write that to a file and post... More soon...

Rainer

Please include the template below in your configuration and let rsyslogd write to a file using this template:

$template DEBUG,"Debug line with all properties:\nFROMHOST: '%FROMHOST%', HOSTNAME: '%HOSTNAME%', PRI: %PRI%,\nsyslogtag '%syslogtag%', programname: '%programname%', APP-NAME: '%APP-NAME%', PROCID: '%PROCID%', MSGID: '%MSGID%',\nTIMESTAMP: '%TIMESTAMP%', STRUCTURED-DATA: '%STRUCTURED-DATA%',\nmsg: '%msg%'\nrawmsg: '%rawmsg%'\n\n"
*.* -/var/log/debug;DEBUG # or whatever file you like

The, please post a few messages. The template provides the raw message received plus some parsed fields and thus enables me to a) get a clear view and b) reproduce the sender's behaviour in lab.

Rainer

The folllowing output :


Debug line with all properties:
FROMHOST: t24_db02, HOSTNAME: Message, PRI: 31,
syslogtag forwarded, programname: forwarded, APP-NAME: forwarded, PROCID: -, MSGID: -,
TIMESTAMP: Dec 2 09:14:18, STRUCTURED-DATA: -,
msg: from t24_db02: nrpe[40826]: Command completed with return code 0 and output: DISK OK - free space: /usr 728 MB (23%);| /usr=2408MB;2822;3104;0;3136
rawmsg: <31>Dec 2 09:14:18 Message forwarded from t24_db02: nrpe[40826]: Command completed with return code 0 and output: DISK OK - free space: /usr 728 MB (23%);| /usr=2408MB;2822;3104;0;3136

Debug line with all properties:
FROMHOST: t24_db02, HOSTNAME: Message, PRI: 31,
syslogtag forwarded, programname: forwarded, APP-NAME: forwarded, PROCID: -, MSGID: -,
TIMESTAMP: Dec 2 09:14:53, STRUCTURED-DATA: -,
msg: from t24_db02: nrpe[17994]: Connection from 10.1.10.100 closed.
rawmsg: <31>Dec 2 09:14:53 Message forwarded from t24_db02: nrpe[17994]: Connection from 10.1.10.100 closed.

Please take a look output above, give me your ideas.

PS: Everything is ok, except to AIX OS.
Thanks a lot!

Please all have a look at this thread: http://www.rsyslog.com/PNphpBB2-viewtopic-t-101.phtml

Rainer

Hi, I have a similiar problem with HP-UX messages getting to rsyslog. I have rsyslog setup on a RHEL 4 Update 5 server (linux hostnames working fine) but when it gets messages from a HP-UX 11i OS server I get this from the debug template:

Debug line with all properties:
FROMHOST: '**INVALID PROPERTY NAME**', HOSTNAME: 'sshd[4442]:', PRI: 38,
syslogtag 'Accepted', programname: '**INVALID PROPERTY NAME**', APP-NAME: '**INVALID PROPERTY NAME**', PROCID: '**INVALID PROPERTY NAME**', MSGID: '**INVALID PROPERTY NAME**',
TIMESTAMP: 'Aug 1 15:03:23', STRUCTURED-DATA: '**INVALID PROPERTY NAME**',
msg: ' password for root from 10.6.1.19 port 2020 ssh2'
rawmsg: '<38>Aug 1 15:03:23 sshd[4442]: Accepted password for root from 10.6.1.19 port 2020 ssh2'

Please help

BTW awesome program!

Hi,

it looks like you are using the stable build. Right now, it is quite outdated (will be re-released in a month or so, based on the current devel version). Please go to the download section and get the most current build. Possibly, it also fixes the problem.

Please let me know the outcome.

Thanks,
Rainer

Still the same problem.

Could you please once again post the output of the debug line? All in all, I noticed that there is not hostname at all in the rawmessage. So rsyslog can not extract it. If you can go with the sender of the host, you can also use the %FROMHOST% (all caps!) property. In NAT and relay, however, this is not the real sender (it is always the last hop).

I've been searching for a solution to this problem. The issue is simply the "Message forwarded from" string that AIX adds to forwarded syslog messages by default. If you run your native AIX syslogd clients with the "-n" flag it will suppress that string. Update syslogd in /etc/rc.tcpip & run chssys and you should be good to go.

Once I did that, rsyslogd began parsing my aix syslog forwards correctly (flat file & mysql format) without the use of any templates.

ah, interesting to see! That's definitely the cause and the cure