filter out certain messages from writing to database

This is the place for you, if you got rsyslog up and running but wonder how to make it do what you want.

Moderator: rgerhards

Re: filter out certain messages from writing to database

Postby rgerhards » Mon Jun 01, 2009 5:42 pm

Have you checked what your hostname looks like? I guess there the problem is rooted.

Side-Note: if you do this work for profit, I would seriously consider our 29 Euro config file service. It's really paying off rather quickly (compare it to the week's worth of effort...). Plus, you help the project ;)

http://www.rsyslog.com/doc-professional_support.html

HTH
Rainer
User avatar
rgerhards
Site Admin
 
Posts: 2647
Joined: Thu Feb 13, 2003 11:57 am

Professional Services Information

  • Custom written rsyslog.conf?
  • Maintenance Contract?
  • Installation support?

Re: filter out certain messages from writing to database

Postby rgerhards » Mon Jun 01, 2009 5:46 pm

Just noticed: you also run the first action asynchronously, so it has no effect. I also suggest to view

http://www.rsyslog.com/Article350.phtml

Rainer
User avatar
rgerhards
Site Admin
 
Posts: 2647
Joined: Thu Feb 13, 2003 11:57 am

Re: filter out certain messages from writing to database

Postby zilogz » Tue Jun 02, 2009 3:21 am

thankyou gerhard

i confirm the hostname strings are the correct case etc, those should match

i watched your whole video (very good) and i understand the concept of asynchronous vs synchronous but i am just having trouble with syntax

what would an example of synchronous processing be with more than one discard filter? what would the config look like for those lines?
i tried:

Code: Select all
changing:

if $hostname startswith 'xlin' then ~
if $hostname startswith 'xlog' then ~
if $hostname startswith 'xprp' then ~
*.* :ommysql:localhost,Syslog,rsyslog,appliance
*.* /u01/logs/master.log

to:

if $hostname startswith 'xlin' then ~
*.* :ommysql:localhost,Syslog,rsyslog,appliance
*.* /u01/logs/master.log
if $hostname startswith 'xlog' then ~
*.* :ommysql:localhost,Syslog,rsyslog,appliance
*.* /u01/logs/master.log
if $hostname startswith 'xprp' then ~
*.* :ommysql:localhost,Syslog,rsyslog,appliance
*.* /u01/logs/master.log


but then i get duplicate messages.. i must be missing something obvious here.. i have read all documentation i can find on the monitorware kb, the wiki, the documentation with the tarball and i have not been able to find an example config of multiple discard filters to one db target.
zilogz
New
 
Posts: 5
Joined: Sun May 31, 2009 5:09 pm

Re: filter out certain messages from writing to database

Postby zilogz » Sun Jun 14, 2009 9:00 am

hello, i was wondering if anyone can still help me with this.. to show a sample if..then code for multiple filters? it would really help me out..

also how can you block the NO MATCH messages when a regex fails?

please help, this is such a great tool.
zilogz
New
 
Posts: 5
Joined: Sun May 31, 2009 5:09 pm

Re: filter out certain messages from writing to database

Postby zilogz » Tue Jun 16, 2009 4:59 pm

i now have this working, albeit it is ugly.
any suggestions to reduce the code, or use a negative matching regex or something would be great!
but it is filtering out these hostnames now:

Code: Select all
$ModLoad ommysql
$ActionQueueType FixedArray
$ActionResumeRetryCount 10

if $hostname != 'xlin004' and \
   $hostname != 'xlin005' and \
   $hostname != 'xlin006' and \
   $hostname != 'xlin007' and \
   $hostname != 'xlin008' and \
   $hostname != 'xlin009' and \
   $hostname != 'xlin010' and \
   $hostname != 'xlin011' and \
   $hostname != 'xlin014' and \
   $hostname != 'xlin016' and \
   $hostname != 'xlin017' and \
   $hostname != 'xlin020' and \
   $hostname != 'xlin024' and \
   $hostname != 'xlin026' and \
   $hostname != 'xlog004' and \
   $hostname != 'xlog005' and \
   $hostname != 'xlog006' and \
   $hostname != 'xlog007' and \
   $hostname != 'xlog008' and \
   $hostname != 'xlog009' and \
   $hostname != 'xlog010' and \
   $hostname != 'xlog011' and \
   $hostname != 'xlog014' and \
   $hostname != 'xlog016' and \
   $hostname != 'xlog017' and \
   $hostname != 'xlog020' and \
   $hostname != 'xlog024' and \
   $hostname != 'xlog026' and \
   $hostname != 'xprp004' and \
   $hostname != 'xprp005' and \
   $hostname != 'xprp006' and \
   $hostname != 'xprp007' and \
   $hostname != 'xprp008' and \
   $hostname != 'xprp009' and \
   $hostname != 'xprp010' and \
   $hostname != 'xprp011' and \
   $hostname != 'xprp014' and \
   $hostname != 'xprp016' and \
   $hostname != 'xprp017' and \
   $hostname != 'xprp020' and \
   $hostname != 'xprp024' and \
   $hostname != 'xprp026' \

then :ommysql:localhost,Syslog,rsyslog,appliance
zilogz
New
 
Posts: 5
Joined: Sun May 31, 2009 5:09 pm

Re: filter out certain messages from writing to database

Postby zilogz » Tue Jun 16, 2009 6:15 pm

ok so i solved my problem above with filtering hostnames (although its big and ugly, if anyone has a more elegant way to do it please let me know)

now, on a different rsyslog server but related to this topic i have a template which rewrites the logs and inserts them into a database

it works great, however when a malformed message comes through, it writes a "NO MATCH" line into the db

how can i go about filtering that out?

Code: Select all

log:

insert into tacacs "swpkabcsmmlsr001,Francis Ancis,fancis,10.92.2.13,quit <cr>,0"
insert into tacacs "gdprabaemmler001,Francis Ancis,fancis,10.92.2.69,quit <cr>,0"
insert into tacacs "clgsabetdcswt099,Cisco Config,cfgback,10.85.242.24,enable <cr>,0"
insert into tacacs "clgsabetdcswt099,Cisco Config,cfgback,10.85.242.24,enable <cr>,0"
insert into tacacs "**NO MATCH**,**NO MATCH**,**NO MATCH**,**NO MATCH**,**NO MATCH**,**NO MATCH**"

template:

$template mysqltemplate,"insert into tacacs (name, username, nas, cmd, priv) VALUES ('%msg:R,ERE,2,DFLT:Access Device=(.*),Real Name=(.*),User-Name=(.*),NAS\-IP\-Address=(.*),cmd=(.*),priv-lvl=(.*),--end%','%msg:R,ERE,3,DFLT:Access Device=(.*),Real Name=(.*),User-Name=(.*),NAS\-IP\-Address=(.*),cmd=(.*),priv-lvl=(.*),--end%','%msg:R,ERE,1,DFLT:Access Device=(.*),Real Name=(.*),User-Name=(.*),NAS\-IP\-Address=(.*),cmd=(.*),priv-lvl=(.*),--end%','%msg:R,ERE,5,DFLT:Access Device=(.*),Real Name=(.*),User-Name=(.*),NAS\-IP\-Address=(.*),cmd=(.*),priv-lvl=(.*),--end%','%msg:R,ERE,6,DFLT:Access Device=(.*),Real Name=(.*),User-Name=(.*),NAS\-IP\-Address=(.*),cmd=(.*),priv-lvl=(.*),--end%')\n",SQL

*.* :ommysql:localhost,tacacs,xxxxx,xxxxx;mysqltemplate

zilogz
New
 
Posts: 5
Joined: Sun May 31, 2009 5:09 pm

Google Ads


Previous

Return to Configuration

Who is online

Users browsing this forum: No registered users and 0 guests

cron