syslog-ng filters and destinations under rsyslog

This is the place for you, if you got rsyslog up and running but wonder how to make it do what you want.

Moderator: rgerhards

syslog-ng filters and destinations under rsyslog

Postby martin_pg » Thu Jun 18, 2009 12:59 pm

Hi all,

I need to find a way to either convert or import syslog-ng.conf files into rsyslog or learn the syntax to classify/store the logfiles depending on the source.

For example, taken from a syslog-ng file:
--
source s_network { udp(); tcp(); };

filter f_systems {
host("1\.2\.3\.\d+");
};

filter f_switches {
host("5\.6\.7\.8|5\.6\.7\.9");
};

destination d_systems {file("/systems.log"); };
destination d_switches {file("/switches.log"); };

log { source(s_network); filter(f_systems); destination(d_systems); };
log { source(s_network); filter(f_switches); destination(d_switches); };
--

I have read the online documentation but I'm unable to find the right answer...

Thanks!
martin_pg
Frequent Poster
 
Posts: 79
Joined: Thu Jul 03, 2008 3:30 pm

Professional Services Information

  • Custom written rsyslog.conf?
  • Maintenance Contract?
  • Installation support?

Re: syslog-ng filters and destinations under rsyslog

Postby rgerhards » Thu Jun 18, 2009 1:00 pm

can you explain what the syslog-ng config does? I have no idea how it works...
User avatar
rgerhards
Site Admin
 
Posts: 2647
Joined: Thu Feb 13, 2003 11:57 am

Re: syslog-ng filters and destinations under rsyslog

Postby martin_pg » Thu Jun 18, 2009 1:28 pm

hi,

source s_network { udp(); tcp(); }; tells syslog-ng to listen on tcp and udp. The rsyslog equivalent would be $UDPServerRun and $InputTCPServerRun .

filter f_switches and destination d_switches are just definitions of source and target files respectively...

The log line can be summarized as: take the logs which you see on either tcp or udp coming from 1.2.3.x and write them to /var/log/mylogs/systems.log - using systems for this example.

cheers
martin_pg
Frequent Poster
 
Posts: 79
Joined: Thu Jul 03, 2008 3:30 pm

Re: syslog-ng filters and destinations under rsyslog

Postby rgerhards » Thu Jun 18, 2009 1:31 pm

not checked the syntax, but along these lines:

:fromhost-ip, isequal, "1.2.3.4" /var/log/mylogs/systems.log
& ~

The tilde deltes the message after it is written.
User avatar
rgerhards
Site Admin
 
Posts: 2647
Joined: Thu Feb 13, 2003 11:57 am

Re: syslog-ng filters and destinations under rsyslog

Postby martin_pg » Thu Jun 18, 2009 1:53 pm

I haven't tried it yet but still have one more question: what if I have a block of source IP addresses?

thanks
martin_pg
Frequent Poster
 
Posts: 79
Joined: Thu Jul 03, 2008 3:30 pm

Re: syslog-ng filters and destinations under rsyslog

Postby rgerhards » Thu Jun 18, 2009 1:57 pm

well, then you either need to duplicate the filters or use a long expression-based filter. The former offers better performance if you anticipate high loads.
User avatar
rgerhards
Site Admin
 
Posts: 2647
Joined: Thu Feb 13, 2003 11:57 am

Re: syslog-ng filters and destinations under rsyslog

Postby martin_pg » Thu Jun 18, 2009 3:01 pm

OK.

Is this correct/safe to say:

a)
:fromhost-ip, startswith, "1.2.3." /var/log/mylogs/systems.log

b)
$template DynaFile,"/var/log/mylogs/%HOSTNAME%.%$DAY%%$MONTH%%$YEAR%.log"
:fromhost-ip, startswith, "1.2.3." -?DynaFile

or

:fromhost-ip, isequal, "1.2.3.4" -?DynaFile



thanks
martin_pg
Frequent Poster
 
Posts: 79
Joined: Thu Jul 03, 2008 3:30 pm

Re: syslog-ng filters and destinations under rsyslog

Postby rgerhards » Thu Jun 18, 2009 3:04 pm

I don't checked the systax, but after a filter comes an action. It doesn't matter what the filter and the action is. So both are valid combinations, as is the third.

Pay attention to rsyslog error messages, if any (I said I did NOT check the syntax ;))
User avatar
rgerhards
Site Admin
 
Posts: 2647
Joined: Thu Feb 13, 2003 11:57 am

Re: syslog-ng filters and destinations under rsyslog

Postby martin_pg » Thu Jun 18, 2009 3:27 pm

OK, I added some filters to test.

Cheking the configuration with the -d -N switches shows the following error:
--
# rsyslogd -c3 -d -N 1 -f rsyslog.conf
rsyslogd: version 3.22.0, config validation run (level 1), master config rsyslog.conf
rsyslogd: CONFIG ERROR: could not interpret master config file 'rsyslog.conf'. [try http://www.rsyslog.com/e/2013 ]
rsyslogd: EMERGENCY CONFIGURATION ACTIVATED - fix rsyslog config file!
rsyslogd: End of config validation run. Bye.
--

Removing the new filters or increasing the -N level (up to 9 so far) yields the same results so I don't know what I've done wrong.
Oddly enough, rsyslog works fine when run w/o the -d -N switches....


thanks
martin_pg
Frequent Poster
 
Posts: 79
Joined: Thu Jul 03, 2008 3:30 pm

Re: syslog-ng filters and destinations under rsyslog

Postby rgerhards » Thu Jun 18, 2009 3:29 pm

I think that's a bug that was fixed in git. If there is no error message, you are fine.
User avatar
rgerhards
Site Admin
 
Posts: 2647
Joined: Thu Feb 13, 2003 11:57 am

Re: syslog-ng filters and destinations under rsyslog

Postby martin_pg » Fri Jun 19, 2009 11:58 am

hi again,

short version: it didn't work.

long version: in the rsyslog server configuration I added the following directives:

$IncludeConfig /etc/rsyslog.d/
$ResetConfigVariables

which come before this:

$template DynaFile,"/var/log/current/%HOSTNAME%.%$DAY%%$MONTH%%$YEAR%.log"
*.* -?DynaFile

...actually, the DynaFile is the last line on the config file...

Under /etc/rsyslog.d I have at least 2 files which look like this (IPs and hostnames changed to protect the innocent :-) )
--
rsyslog.d]# head -4 servers.conf
$template ServerFile,"/var/log/current/servers/%HOSTNAME%.%$DAY%%$MONTH%%$YEAR%.log"
*.* -?ServerFile
:fromhost, isequal, "server1" -?ServerFile
:fromhost, isequal, "server2" -?ServerFile

rsyslog.d]# head -4 networks.conf
$template NetworkFile,"/var/log/current/networks/%HOSTNAME%.%$DAY%%$MONTH%%$YEAR%.log"
*.* -?NetworkFile
:fromhost-ip, isequal, "1.1.1.1" -?NetworkFile
:fromhost-ip, isequal, "1.1.1.2" -?NetworkFile
--

Is my approach correct or have I done anything wrong? :-/


thanks
martin_pg
Frequent Poster
 
Posts: 79
Joined: Thu Jul 03, 2008 3:30 pm

Re: syslog-ng filters and destinations under rsyslog

Postby rgerhards » Fri Jun 19, 2009 1:06 pm

doesn't look too bad ;) What means "did not work"?

One thing I notice is that you do not throw away the messages once you are done with them (the discard action, tilde). so all messages will go to all logs. Is that the problem? If so, add

& ~

(on its OWN line) after the last "write to" action (see my previous example).
User avatar
rgerhards
Site Admin
 
Posts: 2647
Joined: Thu Feb 13, 2003 11:57 am

Re: syslog-ng filters and destinations under rsyslog

Postby martin_pg » Fri Jun 19, 2009 2:16 pm

oops! my bad...

hmm... do I have to write &~ after each :fromhost-ip line or just one at the very end of the config file?


thanks
martin_pg
Frequent Poster
 
Posts: 79
Joined: Thu Jul 03, 2008 3:30 pm

Re: syslog-ng filters and destinations under rsyslog

Postby martin_pg » Fri Jun 19, 2009 3:49 pm

I forgot to mention...

o all messages will go to all logs. Is that the problem?


no.
I was expecting the messages to be written under, for example, /var/log/current/networks but they still end up under /var/log/current ...

cheers
martin_pg
Frequent Poster
 
Posts: 79
Joined: Thu Jul 03, 2008 3:30 pm

Re: syslog-ng filters and destinations under rsyslog

Postby rgerhards » Tue Jun 23, 2009 4:07 pm

did you do a restart after the config change? If so, please post a debug log while messages are written to the wrong location.
User avatar
rgerhards
Site Admin
 
Posts: 2647
Joined: Thu Feb 13, 2003 11:57 am

Google Ads


Next

Return to Configuration

Who is online

Users browsing this forum: No registered users and 0 guests

cron