MonitorWare Knowledge Base

Hi Guys

Ive set up rsyslog on a fairly powerful centos based server.
i throw 500 records at it from a syslog generator an it only captures about 25% of the events each time. each time its a different number. if i put a wait between the sends the capture rate increases but its never 100%

ive inserted the following into /etc/rsyslog.conf but it does not seem to make a difference

#$WorkDirectory /usr/local/work
#$ActionQueueType LinkedList # use asynchronous processing
#$ActionQueueFileName dbq # set file name, also enables disk mode assist
#$ActionResumeRetryCount -1 # infinite retries on insert failure, default is 0, -1 is eternal

Any ideas?

• Custom written rsyslog.conf?
• Maintenance Contract?
• Installation support?

speedfox wrote:#$WorkDirectory /usr/local/work
#$ActionQueueType LinkedList # use asynchronous processing
#$ActionQueueFileName dbq # set file name, also enables disk mode assist
#$ActionResumeRetryCount -1 # infinite retries on insert failure, default is 0, -1 is eternal

These are commented out but would not make any difference if they were active.

Please provide your full rsyslog.conf plus all include files. Chances are great that the sender overruns the (local sender) UDP stack, in which case UDP silently drops messages. But this is guesswork, I need to see the config first. Are sender and receiver on the same host?

Rainer

here is the config file

they are on the same network.

$ModLoad ommysql #provides logging to mysql
$ModLoad immark # provides --MARK-- message capability
$ModLoad imudp # provides UDP syslog reception
$ModLoad imtcp # provides TCP syslog reception and GSS-API (if compiled to support it)
$ModLoad imuxsock # provides support for local system logging (e.g. via logger command)
$ModLoad imklog # provides kernel logging support (previously done by rklogd)

#for high volume servers
#$WorkDirectory /usr/local/work
#$ActionQueueType LinkedList # use asynchronous processing
#$ActionQueueFileName dbq # set file name, also enables disk mode assist
#$ActionResumeRetryCount -1 # infinite retries on insert failure, default is 0, -1 is eternal


# ######### Receiving Messages from Remote Hosts ##########
# TCP Syslog Server:
# provides TCP syslog reception and GSS-API (if compiled to support it)
$InputTCPServerRun 514 # start up TCP listener at port 514

# UDP Syslog Server:
$UDPServerRun 514 # start a UDP syslog server at standard port 514



# Log all kernel messages to the console.
# Logging much else clutters up the screen.
#kern.* /dev/console

# Log anything (except mail) of level info or higher.
# Don't log private authentication messages!

#dont use the default inserts, they are buggy, see below
#*.* :ommysql:127.0.0.1,Syslog,syslog,password

*.* /var/log/messages


#CISCO (PIX ASA) and Checkpoint #Modified default template from /tools/syslogd.c
#$template SystemEvent, "insert into SystemEvents (Message, Facility, FromHost, sport, tohost, dport, Priority, MessageID, DeviceReportedTime, ReceivedAt, InfoUnitID, SysLogTag) values ('%msg%', %syslogfacility%, '%HOSTNAME%','%msg:R,ERE,1,BLANK,0:from.*/(.*) to--end%','%msg:R,ERE,1,BLANK,0:to (\b(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\b)--end%','%msg:R,ERE,1,BLANK:to.*\/(.*) flag--end%', %syslogpriority%,'%msg:9:14%', '%timereported:::date-mysql%', '%timegenerated:::date-mysql%', %iut%, '%syslogtag%')",SQL
$template SystemEvent, "insert into SystemEvents (hostname,Message, Facility, FromHost, sport, tohost, dport, Priority, MessageID, DeviceReportedTime, ReceivedAt, InfoUnitID, SysLogTag) values ('%HOSTNAME%','%msg%', %syslogfacility%, '%msg:R,ERE,1,BLANK,0:outside:(\b(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\b)--end%','%msg:R,ERE,2,BLANK,0:(outside:|3rdParty:)[^ ]+?/([^ ]+?)\s--end%','%msg:R,ERE,1,BLANK,0:inside:(\b(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\b)--end%','%msg:R,ERE,2,BLANK:(inside:|dmz1:|dmz2:)[^ ]+?/([^ ]+?)\s--end%', %syslogpriority%,'%msg:R,ERE,1,BLANK:%......(.+?): --end%', '%timereported:::date-mysql%', '%timegenerated:::date-mysql%', %iut%, '%syslogtag%')",SQL
if $msg contains 'PIX' or $msg contains 'ASA' then :ommysql:192.148.2.58,logger,logger,blah3;SystemEvent




# The authpriv file has restricted access.
authpriv.* /var/log/secure

# Log all the mail messages in one place.
mail.* -/var/log/maillog


# Log cron stuff
cron.* /var/log/messages

# Everybody gets emergency messages
*.emerg *

# Save news errors of level crit and higher in a special file.
uucp,news.crit /var/log/spooler

# Save boot messages also to boot.log
local7.* /var/log/boot.log

#
# INN
#
news.=crit /var/log/news/news.crit
news.=err /var/log/news/news.err
news.notice /var/log/news/news.notice

MODERATOR EDIT: turned off smilies for better readibility

Do you use UDP or TCP from the PIX? Cisco IOS has a broken syslog/tcp implementation (search forum), so this may be the root cause if using TCP.

ahh that might be why. i think im using tcp.

cool thanks.

how would i change from using tcp to udp?

speedfox wrote:how would i change from using tcp to udp?

# Provides UDP syslog reception
$ModLoad imudp.so
$UDPServerRun 514

I believe those 2 lines in rsyslog.conf should enable it. At least according to what I have read.

yes, that's right. Plus, you should think about your local firewall (on the system that runs rsyslog). Often, port 514 udp is blocked. And finally, you need to change the PIX config

Rainer

rgerhards wrote:yes, that's right. Plus, you should think about your local firewall (on the system that runs rsyslog). Often, port 514 udp is blocked. And finally, you need to change the PIX config

Rainer

You know what.. I bet that's my problem.. And what's retarded is thats the SAME problem I have every time I set up a new network service and I always forget to check it lol..