Syslog messages do not conform to RFC behaviour

General discussions here

Moderator: rgerhards

Google Ads


Syslog messages do not conform to RFC behaviour

Postby cybersec » Thu Oct 12, 2017 11:17 pm

Hi,

Newbie to the world of syslog and rsyslogd so apologies in advance for the rsyslog 101 style questions!

I have a bunch of log sources (i.e devices) sending what they claim are syslog messages to a central rsyslog relay server (v7.4.7) which is configured to forward all events to a commercial SIEM tool. What I would like to validate is the following:

a) As per RFC 3164, if a log source (device) does not send a message that conforms to RFC 3164, the rsyslog relay server should transform the message so it does confirm? Is the same behaviour expected for messages that do not conform to RFC 5424?

b) If the behaviour above is true, does this not cause issues with parsing/moralisation when forwarding to a commercial SIEM tool (e.g. IBM QRadar, AlientVault USM, McAfee ESM etc) as the SIEM is unable to determine the log source type?

c) Can one claim message integrity with this 'transform-to-conform-to-rfc' behaviour'?

Thanks!
cybersec
New
 
Posts: 8
Joined: Thu Oct 12, 2017 11:01 pm

Urgent Question?

  • Pulling out your Hair?
  • Wasting Time and Money?
  • Deadline Approaching?

Re: Syslog messages do not conform to RFC behaviour

Postby dlang » Mon Oct 16, 2017 7:04 am

To answer you last question first.

I spent almost 17 years working in Internet Banking supporting almost 2000 banks and credit unions (so LOTS of audit oversight)

None of these audits from the banks, credit unions, or the federal oversight groups had any problem with logs being transformed in a pre-determined way as they were processed.

The "make sure the logs haven't been changed" is worrying about the logs being tampered with to remove evidence. As long as your logs are processed by tools, and you can show the configs (to make sure there are no configs that say "if user ='hacker' change things") nobody has any problems using the modified logs.

This is even when you go as far as to parse the logs and send the results in a normalized JSON format.

No to your other questions.

If rsyslog receives a log, it does it's best to extract meaning from the log, and what it sends out using the standard templates will be compliant.

The rfc5424 is fairly strict about compliance, it's a new standard and anyone claiming to be using it should use it correctly.

The rfc3164 parser is a 'last resort' parser, it tries _very_ hard to 'do the right thing', and has decades of heuristics baked into it to work around various broken logs that it gets. That doesn't mean that the heuristics always figure out the best way to parse the logs, and so sometimes you need to detect the broken logs and fix them yourself (frequently a output template is all you need, sometimes you need to do more)
dlang
Frequent Poster
 
Posts: 1001
Joined: Mon Sep 15, 2008 7:44 am

Re: Syslog messages do not conform to RFC behaviour

Postby cybersec » Mon Oct 16, 2017 9:58 am

Thanks dlang.

I agree with your transformation position, if you can prove the transformation is robust there should be no problems (e.g. accurate, repeatable etc).

However, out of interest, is there a way to configure rsyslogd (relay) to just forward what is receives without transformation?

I've been looking at $ActionForwardDefaultTemplate but cannot figure out what the default template is under rsyslogd v7.4.7

Thanks!
cybersec
New
 
Posts: 8
Joined: Thu Oct 12, 2017 11:01 pm

Re: Syslog messages do not conform to RFC behaviour

Postby dlang » Mon Oct 16, 2017 10:22 am

yes, you could make a template $rawmsg

but that actually violates the RFC standards that say that a relay should clean some things up :-)

What I do is I have the relay turn anything it receives into a JSON message and add metadata to the message saying what IP the log was received from, when it was received, hat relay handled it, etc. At the final receiver, I can recreate the original message if needed, or I can use the metadata.
dlang
Frequent Poster
 
Posts: 1001
Joined: Mon Sep 15, 2008 7:44 am

Re: Syslog messages do not conform to RFC behaviour

Postby cybersec » Mon Oct 16, 2017 10:38 am

Thanks dlang.

Very helpful. Time for me to power on some virtual machines and get testing.
cybersec
New
 
Posts: 8
Joined: Thu Oct 12, 2017 11:01 pm

Google Ads



Return to General

Who is online

Users browsing this forum: No registered users and 1 guest

cron