MonitorWare Knowledge Base

I'm still looking into this a bit more on my own, but I figured I'd ask about it here as well....

I'm getting syslogs from some netbsd 1.6 hosts which are not being formatted correctly. messages from another field (not hostname ) are showing up as the hostname...

for example, a plaintext message on the netbsd host shows up as the following (spaces inply change of field)

Aug 30 04:50:00 host1 /usr/sbin/cron[28464]: (root) CMD FINISH (/usr/libexec/atrun)

is showing up as this in the db as:


+-----------------------+----------+----------+-------+--------+------------+----------+---------+----------------------------------+-----+
| host | facility | priority | level | tag | date | time | program | msg | seq |
+-----------------------+----------+----------+-------+--------+------------+----------+---------+----------------------------------+-----+
| /usr/sbin/cron[1193]: | 9 | 6 | NULL | (root) | 2005-08-30 | 04:40:00 | NULL | CMD START (/usr/libexec/atrun) | 643 |
+-----------------------+----------+----------+-------+--------+------------+----------+---------+----------------------------------+-----+


any ideas?

• Custom written rsyslog.conf?
• Maintenance Contract?
• Installation support?

mmmhhh... that's strange. Did you get the message from a packet dump or where did you receive it from? Also, could you please post the output of "rsyslog -v".

Thanks - and a good new year

Rainer

On the NetBSD host I have the 'forward syslog' entry as my lastline, so syslogs should be exist locally and forward as well. I compared the MySQL entry to the corresponding one in the flatfiles.


output:

[root@myhost]# /usr/sbin/rsyslogd -v
rsyslogd 1.12.1, compiled with:
FEATURE_PTHREADS (dual-threading)
FEATURE_REGEXP
FEATURE_DB
FEATURE_LARGEFILE
SYSLOG_INET (Internet/remote support)


~Thanks

mmmhhh... Can you please send me the debug output (syslogd -n -d) when such a remote mesage is received. That will help me to see what exactly is going on...

The file data is unfortunately not exactly what is seen on the wire, but the debug output will tell us

Thanks,
Rainer

FROM DEBUG OUTPUT:

-1208027456: Message from UDP inetd socket: #11, host: 10.10.36.44
-1208027456: Message length: 82, File descriptor: 11.
-1208027456: logmsg: cron.info<78>, flags 2, from (null), msg Sep 3 03:00:00 /usr/sbin/cron[7587]: (root) CMD START (/usr/bin/newsyslog)
-1208027456: EnqueueMsg signaled condition (0)
-1208030288: -1208027456: Lone worker is running...
Listening on syslog UDP port.
-1208027456: Called fprintlog, logging to MYSQL

MYSQL ENTRY:

mysql> select * from logs where seq="77574";
+-----------------------+----------+----------+-------+--------+------------+----------+---------+----------------------------------+-------+
| host | facility | priority | level | tag | date | time | program | msg | seq |
+-----------------------+----------+----------+-------+--------+------------+----------+---------+----------------------------------+-------+
| /usr/sbin/cron[7587]: | 9 | 6 | NULL | (root) | 2006-09-03 | 03:00:00 | NULL | CMD START (/usr/bin/newsyslog) | 77574 |
+-----------------------+----------+----------+-------+--------+------------+----------+---------+----------------------------------+-------+


perhaps you can explain the output better. but it looks as if the netbsd hosts are sending a blank hostname when newsyslog is being run? Causing the msg to be the hostname? Thanks.

Hi,

yes, BSD syslogd in general does not send any hostname. This was discussed some time ago and I included patches to smartly handle that situation. I have used your test message in my environment and it correctly identifies the TAG as TAG and detects the missing hostname. It is really strange that you do not get the same behaviour with the 1.12.1 version. I also know from other users that this functionality works.

Could you please download the absolutely newest snapshot from the anonymous CVS on sourceforget and rebuild it. With that, we would at least run the exact same code. It is really strange...

Rainer

Sourceforege's WebCVS doesnt seem to be working, and I'm having difficulty accessing your project's space in CVS (denied errors) as well.....

I dont suppose there is a 'nightly build' somewhere?

Sorry for the late reply. I guess starting from now I am again able to reply in a timely manner. There are no automatically build nightly tarballs, but I have done one manually. It is available at

http://download.adiscon.com/rsyslog/rsy ... pre.tar.gz

Please give it a try. If it does not work, please post debug output (I'd prefer around 50 lines of it or two complete incomming messages).

Rainer

Thanks, I will try this and post results as soon as possible.



Also, thanks again for all your help; rsyslog is really great and the support you provide is just as usefull.

After upgrading to the above supplied 1.12.2-pre I still see the same results.
Syslog messages are being received from remotely from server 'servername' but the programname filter does not match 'sendmail':

Code: Select all

-1208006048: ----------------------------------------
Calling select, active file descriptors (max 5): 3 5
-1208009808: Lone worker is running...
-1208009808: programname filter 'sendmail' does not match ''
-1208009808: programname filter 'sendmail' does not match ''
-1208009808: Called fprintlog, logging to FILE (/tmp/mail)
-1208009808: Lone worker is running...
-1208009808: programname filter 'sendmail' does not match ''
-1208009808: programname filter 'sendmail' does not match ''
-1208009808: Called fprintlog, logging to FILE (/tmp/mail)
-1208009808: singleWorker: queue EMPTY, waiting for next message.
-1208006048:
Successful select, descriptor count = 1, Activity on: 5
-1208006048: Message from UDP inetd socket: #5, host: W.X.Y.X
-1208006048: Message length: 111, File descriptor: 5.
-1208006048: logmsg: mail.notice<21>, flags 2, from 'remoteserver', msg sendmail[18205]: k0A4ec1t018205: lost input channel from [61.173.23.51] to remoteserver.tpgi.com.au after rcpt

-1208006048: Message has legacy syslog format.
-1208006048: EnqueueMsg signaled condition (0)
-1208006048: Listening on syslog UDP port.
-1208006048: ----------------------------------------
Calling select, active file descriptors (max 5): 3 5
-1208006048:
Successful select, descriptor count = 1, Activity on: 5
-1208006048: Message from UDP inetd socket: #5, host: W.X.Y.X
-1208009808: Lone worker is running...
-1208009808: programname filter 'sendmail' does not match ''
-1208009808: programname filter 'sendmail' does not match ''
-1208009808: Called fprintlog, logging to FILE (/tmp/mail)
-1208006048: Message length: 161, File descriptor: 5.
-1208006048: logmsg: mail.info<22>, flags 2, from 'remoteserver', msg sendmail[18205]: k0A4ec1t018205: from=<ardolph@DOMAIN>, size=0, class=0, nrcpts=0, proto=SMTP, daemon=remoteserver.tpgi.com.au, relay=forign.server.tld

-1208006048: Message has legacy syslog format.
-1208006048: EnqueueMsg signaled condition (0)
-1208006048: Listening on syslog UDP port.
1208006048: ----------------------------------------
Calling select, active file descriptors (max 5): 3 5
-1208006048:
Successful select, descriptor count = 1, Activity on: 5
-1208006048: Message from UDP inetd socket: #5, host: W.X.Y.X
-1208006048: Message length: 169, File descriptor: 5.
-1208006048: logmsg: mail.info<22>, flags 2, from 'remoteserver', msg sendmail[18391]: k0A4eeJk018391: Milter: to=<byrne@DOMAIN>, reject=450 4.7.1 Greylisted (see http://isg.ee.ethz.ch/tools/postgrey/help/tpg.com.au.html)

-1208006048: Message has legacy syslog format.
-1208006048: EnqueueMsg signaled condition (0)
-1208006048: Listening on syslog UDP port.
-1208006048: ----------------------------------------
Calling select, active file descriptors (max 5): 3 5
-1208009808: Lone worker is running...
-1208009808: programname filter 'sendmail' does not match ''
-1208009808: programname filter 'sendmail' does not match ''
-1208009808: Called fprintlog, logging to FILE (/tmp/mail)
-1208009808: Lone worker is running...
-1208009808: programname filter 'sendmail' does not match ''
-1208009808: programname filter 'sendmail' does not match ''
-1208009808: Called fprintlog, logging to FILE (/tmp/mail)
-1208009808: singleWorker: queue EMPTY, waiting for next message.
-1208006048:
Successful select, descriptor count = 1, Activity on: 5
-1208006048: Message from UDP inetd socket: #5, host: W.X.Y.X
-1208006048: Message length: 148, File descriptor: 5.
-1208006048: logmsg: mail.notice<21>, flags 2, from 'remoteserver', msg sendmail[18391]: k0A4eeJk018391: lost input channel from c-24-34-119-105.hsd1.ma.comcast.net [24.34.119.105] to remoteserver.tpgi.com.au after rcpt

-1208006048: Message has legacy syslog format.
-1208006048: EnqueueMsg signaled condition (0)
-1208006048: Listening on syslog UDP port.
-1208006048: ----------------------------------------
Calling select, active file descriptors (max 5): 3 5
-1208009808: Lone worker is running...
-1208009808: programname filter 'sendmail' does not match ''
-1208009808: programname filter 'sendmail' does not match ''
-1208009808: Called fprintlog, logging to FILE (/tmp/mail)
-1208006048:
Successful select, descriptor count = 1, Activity on: 5
-1208006048: Message from UDP inetd socket: #5, host: W.X.Y.X
-1208006048: Message length: 188, File descriptor: 5.
-1208006048: logmsg: mail.info<22>, flags 2, from 'remoteserver', msg sendmail[18391]: k0A4eeJk018391: from=<thomas@DOMAIN>, size=0, class=0, nrcpts=0, proto=SMTP, daemon=remoteserver.tpgi.com.au, relay=forign.server.tld

-1208006048: Message has legacy syslog format.
-1208006048: EnqueueMsg signaled condition (0)
-1208006048: Listening on syslog UDP port.
-1208006048: ----------------------------------------
Calling select, active file descriptors (max 5): 3 5
-1208009808: Lone worker is running...
-1208009808: programname filter 'sendmail' does not match ''
-1208009808: programname filter 'sendmail' does not match ''
-1208009808: Called fprintlog, logging to FILE (/tmp/mail)
-1208009808: singleWorker: queue EMPTY, waiting for next message.
-1208006048:
Successful select, descriptor count = 1, Activity on: 5
-1208006048: Message from UDP inetd socket: #5, host: W.X.Y.X
-1208006048: Message length: 150, File descriptor: 5.
-1208006048: logmsg: mail.info<22>, flags 2, from 'remoteserver', msg sendmail[18356]: k0A4eaUV018356: CPE00b0d01792fb-CM014370003644.cpe.net.cable.rogers.com [72.138.121.14]: Possible SMTP RCPT flood, throttling.

-1208006048: Message has legacy syslog format.
-1208006048: EnqueueMsg signaled condition (0)
-1208006048: Listening on syslog UDP port.
-1208009808: -1208006048: Lone worker is running...
----------------------------------------
Calling select, active file descriptors (max 5): -1208006048: 3 5 programname filter 'sendmail' does not match ''

-1208009808: programname filter 'sendmail' does not match ''
-1208009808: Called fprintlog, logging to FILE (/tmp/mail)
-1208009808: singleWorker: queue EMPTY, waiting for next message.
-1208006048:
Successful select, descriptor count = 1, Activity on: 5
-1208006048: Message from UDP inetd socket: #5, host: W.X.Y.X
-1208006048: Message length: 97, File descriptor: 5.
-1208006048: logmsg: mail.info<22>, flags 2, from 'remoteserver', msg sendmail[18399]: NOQUEUE: connect from c-24-34-119-105.hsd1.ma.comcast.net [24.34.119.105]

-1208006048: Message has legacy syslog format.
-1208006048: EnqueueMsg signaled condition (0)
-1208006048: Listening on syslog UDP port.
-1208009808: -1208006048: Lone worker is running...
----------------------------------------
Calling select, active file descriptors (max 5): 3 5
-1208009808: programname filter 'sendmail' does not match ''
-1208009808: programname filter 'sendmail' does not match ''
-1208009808: Called fprintlog, logging to FILE (/tmp/mail)
-1208009808: singleWorker: queue EMPTY, waiting for next message.
-1208006048:
Successful select, descriptor count = 1, Activity on: 5
-1208006048: Message from UDP inetd socket: #5, host: W.X.Y.X
-1208006048: Message length: 83, File descriptor: 5.
-1208006048: logmsg: mail.info<22>, flags 2, from 'remoteserver', msg sendmail[18383]: k0A4edTH018383: subject=.NOTIFICATION.-.Shareholder.Release

-1208006048: Message has legacy syslog format.
-1208006048: EnqueueMsg signaled condition (0)
-1208006048: Listening on syslog UDP port.
-1208006048: ----------------------------------------
Calling select, active file descriptors (max 5): 3 5
-1208009808: Lone worker is running...
-1208009808: programname filter 'sendmail' does not match ''
-1208009808: programname filter 'sendmail' does not match ''
-1208009808: Called fprintlog, logging to FILE (/tmp/mail)
-1208009808: singleWorker: queue EMPTY, waiting for next message.
-1208006048:
Successful select, descriptor count = 1, Activity on: 5
-1208006048: Message from UDP inetd socket: #5, host: W.X.Y.X
-1208006048: Message length: 233, File descriptor: 5.
-1208006048: logmsg: mail.info<22>, flags 2, from 'remoteserver', msg sendmail[18383]: k0A4edTH018383: from=<tzfolr@DOMAIN>, size=2214, class=0, nrcpts=1, msgid=<07457518765.20060110153517@DOMAIN>, proto=SMTP, daemon=remoteserver.tpgi.com.au, relay=forign.server.tld

-1208006048: Message has legacy syslog format.
-1208006048: EnqueueMsg signaled condition (0)
-1208006048: Listening on syslog UDP port.
-1208006048: ----------------------------------------
Calling select, active file descriptors (max 5): 3 5
-1208009808: Lone worker is running...
-1208009808: programname filter 'sendmail' does not match ''
-1208009808: programname filter 'sendmail' does not match ''
-1208009808: Called fprintlog, logging to FILE (/tmp/mail)
-1208009808: singleWorker: queue EMPTY, waiting for next message.
-1208006048:
Successful select, descriptor count = 1, Activity on: 5
-1208006048: Message from UDP inetd socket: #5, host: W.X.Y.X
-1208006048: Message length: 153, File descriptor: 5.


OK, I can see what is the cause with the messages of the last debug log. They lack the timestamp information (not only the sender host), this confuses the messages parser. I could reproduce the problem . I'll have a look at how to fix that and will post when done.

Rainer

OK, actually this was easier than I thought

Open syslogd.c in your editor. Go to line 4356. It should read

Code: Select all

bContParse = 0


Delete this line and run make again. That should take care of the problem. As it looks, the contParse setting is a left-over from previous, less smart parser versions. Anyhow, I'll do a full check if there is any evil in deleting it, but I am already 99.999% sure this is the right cure.

Rainer

Yes that fixed it. Removing bContParse did make it start to work.

I am glad it works. I just wonder if plastica's problem is also solved (I could not reproduce his/her problem, thus I am especially curios ).

Rainer

Hello,

I was having the same problems as plastica. But I downloaded the pre-release you posted, and made the changes as suggested, and it seems to be working fine for me now. Just FWIW, thanks.

By the way, rsyslogd compiles cleanly on an OpenBSD 3.7 machine, using the freebsd make.