MonitorWare Knowledge Base

fromhost-ip was introduced in 3.19.2

• Custom written rsyslog.conf?
• Maintenance Contract?
• Installation support?

Hi there again,

I've set up a RHEL 4 with rsyslog v3.20.2 and I've been sending it arkoon syslog messages

Problem is, my filters have a strange comportment.
I've first tried with the complete rule I'd though I'd need, which is

Code: Select all

if $msg contains 'Alerts:' and ($fromhost-ip == 'x.y.z.1' or $fromhost-ip == 'x.y.z.2' or $fromhost-ip == 'x.y.z.3') then /var/log/Alerts_Logs

This one never logged anything so far.

I've tried 2 simpler, just to see if it was my rule that had a problem

Code: Select all

if $msg contains 'Alerts:' then /var/log/Alerts/logs
if $fromhost-ip == 'x.y.z.2' then /var/log/x.y.z.2/logs

these 2 never logged anything either

I've then tried with

Code: Select all

:fromhost-ip, isequal, "x.y.z.1"
*.*   /var/log/x.y.z.1/logs

there I have error messages in the log file, like

the last error occured in /etc/rsyslog.conf, line 81
warning: selector line without actions will be discarded

I've run a debug, just to see if I could see something. I've found this:

4515.921959000:main thread: Action 0x9cad3c8: queue 0x9cad460 created
4515.921985000:main thread: cfline: ':fromhost-ip, isequal, "x.y.z.1"'
4515.921995000:main thread: selector line successfully processed
4515.922005000:main thread: - property-based filter
4515.922021000:main thread: tried selector action for builtin-file: -2001
4515.922032000:main thread: tried selector action for builtin-fwd: -2001
4515.922042000:main thread: tried selector action for builtin-shell: -2001
4515.922053000:main thread: tried selector action for builtin-discard: -2001
4515.922080000:main thread: tried selector action for builtin-usrmsg: -2001
4515.922091000:main thread: config line NOT successfully processed
4515.922103000:main thread: Called LogError, msg: the last error occured in /etc/rsyslog.conf, line 81
4515.922203000:main thread: cfline: '*.* /var/log/x.y.z.1/logs'
4515.922215000:main thread: Called LogError, msg: warning: selector line without actions will be discarded

I'm at a loss of what to do to make my filters work.

As a sidenote, I've tried to add $AllowedUser UDP x.y.z.1, and there rsyslog log something (only with the property-based filter), but the error message directly above is still there (and if I put many property based filters, each logs messages from the AllowedSender still with the errors). Don't know why either.

Any ideas ?

Thorgull wrote:

Code: Select all

:fromhost-ip, isequal, "x.y.z.1"
*.*   /var/log/x.y.z.1/logs

The first line there is a selector without an action. I think what you really want is:

Code: Select all

:fromhost-ip, isequal, "x.y.z.1"    /var/log/x.y.z.1/logs


If that doesn't show what you're looking for, use a rule like this to troubleshoot:

Code: Select all

$template test, "TIME GEN: %timegenerated:::date-rfc3339% HOSTNAME: %hostname% FROMHOST: %fromhost% FROMIP: %fromhost-ip% FACILITY: %syslogfacility% PRIORITY: %syslogpriority% TAG: %syslogtag% MSG: %msg%\n"
*.*   /var/log/test.log;test


-HKS

hkspvt wrote:The first line there is a selector without an action. I think what you really want is:

Code: Select all

:fromhost-ip, isequal, "x.y.z.1"    /var/log/x.y.z.1/logs


oh great, this one worked
I think some precisions on the filter conditions page could be great. I don't know if I'm the only one, but it seems I did not understand how it worked properly with the actual contents only ^^'

Still have to make the others work, but still it's a progress
Many thanks

Hi there again,

I think I've solved all my actual problems, thanks to that litle bit of code from HKS

hkspvt wrote:

Code: Select all

$template test, "TIME GEN: %timegenerated:::date-rfc3339% HOSTNAME: %hostname% FROMHOST: %fromhost% FROMIP: %fromhost-ip% FACILITY: %syslogfacility% PRIORITY: %syslogpriority% TAG: %syslogtag% MSG: %msg%\n"
*.*   /var/log/test.log;test


Thanks to that I've learned that the word I was looking for in $msg was in fact the $syslogtag ^^'

@Rainer: So far, it looks like rsyslog deals correctly with arkoon syslog messages. There is still need for more tests and more time to be sure, but I thought you would like to know

Thanks HKS
Thanks Rainer

I'll keep you up to date if something new happens

Thanks and please keep us posted

Rainer

Hi, back already ^^',

I've encoutered something strange while testing rsyslog.
I've been redirecting logs from some of our most verbose Arkoon to test my server and, more specifically, the cpu load.
Problem is, when I do "top", rsyslog resolutly stays at 0% cpu usage while the 4 cpus each are between 2 and 30%, and no other process seem to be the source of cpu usage.
Everything seems to be loged correctly, however.

It's not a rsyslog problem (so I hope) but it's annoying because one of the purpose of this test server is to show to my boss that rsyslog could replace our old syslog-ng server because it can use more than one cpu.

Would you have an idea about the issue, or a roundabout to know what the cpu usage of rsyslog is wihtout using "top" ?

Thanks in advance.

mhhh... that's strange. Have you enabled thread view in top?

I've installed htop, my version of top seems too old, it does not have the view thread option. I think that's it.

sorry for the disturbance ^^'

my pleasure... and as a side-note, I did not know htop, tried it and really love it Thanks! Rainer

Glad you liked it

I have a question that will maybe sound dumb ^^'
I was wondering how you are seeing the number of messages per second(/minutes/etc) your server is processing.
Is there a "magic" tool that I missed, or some config modifications to do?
I'm asking because doing it the "hard" way (calculations, etc) is scaring me just a bit

+1 - I think this is a good suggestion. And, well, I think it is not yet there

Ok, no problem, I've moved on over this part.

Actually, I have something new to ask ^^'

I've added the exact same sources to my rsyslog server as the actual syslog-ng server, to compare the performances (~40 firewalls, including the slave when there is one).

Globally, rsyslog is still more interesting for our needs, however it looked like rsyslog used more cpu and memory than syslog-ng, with the same logic for the filters.
My colleague working on the syslog-ng told me it was because syslog-ng's host() filter function is taking the IP directly(in the binary code, if I recall correctly) while rsyslog's fromhost-ip is doing a regex on a string (the log message), which takes more memory and processing power.

Well, I've not fully understood his explanations, so it may be a little hazy to you (it is to me ^^'), but what we want to know is if there is another, more optimized, parameter than fromhost-ip to do filters based on IP,
or if the versions following the 3.20.2 stable (devell and such) have an improvement for such filters (like in syslog-ng).

I hope this is understandable enough

Thanks

Well, I guess you colleague was guessing We take the IP address directly from the socket layer, no regexp at all. I can't compare the memory footprints and CPU, because I've never (really ) actually run syslog-ng. In any case, performance is the big theme for v4 and 4.1.4 (the current devel) already performs much faster than v3.

If you post your rsyslog.conf, I can also check if there is some room for optimization.

Thinking about what you describe, the additional memory may be a result from the fact that rsyslog works with real properties, whereas the syslog-ng v2 (as far as Baszi wrote) does not work on properties but only on the message string itself. This may be the reason it uses less CPU & memory. There may be others, too. For example, rsyslog uses multiple threads, depending on machine and setup this can result in more overhead. Oh... I see you use databases. As far as I know, syslog-ng (v2, which seems to be what is currently used) does not support databases directly. So the CPU and memory for database access is not accounted to the syslog-ng process but to the external database writer (that clumpsy push to the pipe and hope for the best trick... ). But now I am guess. Should stop doing so...

I hope that still helps.

Rainer

He was guessing, yes, I forgot the "probably" between "it was" and "because" ^^'

Good to hear, then. I'll give a try to the 4.1.4.

I've joined a sample of my conf, but for reasons I think you'll understand, I can't upload the real one
But this is representative enough of the overall configuration

95% of the conf have sets of rules similar to the FW-AK1's one. The two others are special cases.

Oh, and neither rsyslog nor syslog-ng are logging to a database

thanks