Message Parser --> order of appliance

You need additional help with phplogcon, then write into this forum.

Message Parser --> order of appliance

Postby DocBrown » Thu Nov 20, 2008 3:11 pm

In the data source definition it is possible to assign multiple Message Parsers by comma seperation.

Now lets assume the following situation. There are logs of two different formats written into the SystemEvents table.
One coming from apache and the other from log4J and two message parsers are assigned.

How does phplogcon now recognise which parser it has to assign to which message??

Any hint for me?
DocBrown
Avarage
 
Posts: 18
Joined: Wed Nov 12, 2008 12:58 pm

phpLogCon professional services

  • Custom phpLogCon configuration?
  • Maintenance Contract?
  • Installation support?

Re: Message Parser --> order of appliance

Postby rgerhards » Thu Nov 20, 2008 3:12 pm

as far as I know, they are tried one after another until one succeeds.
User avatar
rgerhards
Site Admin
 
Posts: 2639
Joined: Thu Feb 13, 2003 11:57 am

Re: Message Parser --> order of appliance

Postby alorbach » Thu Nov 20, 2008 3:37 pm

Exactly. If the first parser has a match, it will parse. If the second parser has a match, it will parse. The parsers are processed one by one. But note that each message parsers will increase the php performance usage. So you might consider to split the differrent messagesources into different tables on the receiver side.

regards,
Andre
User avatar
alorbach
Site Admin
 
Posts: 1127
Joined: Thu Feb 13, 2003 11:55 am

Re: Message Parser --> order of appliance

Postby DocBrown » Thu Nov 20, 2008 3:43 pm

That sounds sensible deviding them into multiple database tables.

This leads to the next question, i.e. is it possible to use customised tables for storing log or does phplogcon require a specific table layout.

I once tried a table defined by my self, but it was not possible to display it with phplogcon.
DocBrown
Avarage
 
Posts: 18
Joined: Wed Nov 12, 2008 12:58 pm

Re: Message Parser --> order of appliance

Postby alorbach » Thu Nov 20, 2008 3:56 pm

Well technically it would be possible to extend the table mappings in phpLogCon manually.
If you take a look into include/constants_logstream.php, at the end you will find the default table mappings.
You can copy one of these table mappings, and add it into the end of your config.php, for example:

Code: Select all
$dbmapping['mylayout'][SYSLOG_UID] = "myid";
$dbmapping['mylayout'][SYSLOG_DATE] = "date";
$dbmapping['mylayout'][SYSLOG_HOST] = "sourcefield";
$dbmapping['mylayout'][SYSLOG_MESSAGE] = "msg";


mylayout would be the name of the table mapping you configure in your logstream source, SYSLOG_UID for example is the internal phpLogCon field id and "myid" the name of your custom database table.

Hope this helps,
Andre
User avatar
alorbach
Site Admin
 
Posts: 1127
Joined: Thu Feb 13, 2003 11:55 am

Re: Message Parser --> order of appliance

Postby rgerhards » Thu Nov 20, 2008 5:15 pm

Andre, isn't the support for php-syslog-ng database schema build via a similar way? I don't remember the details, but this here sounds pretty much the same...

Rainer
User avatar
rgerhards
Site Admin
 
Posts: 2639
Joined: Thu Feb 13, 2003 11:57 am

Re: Message Parser --> order of appliance

Postby alorbach » Thu Nov 20, 2008 5:19 pm

Yes, both the monitorware and syslog-ng database schema are defined in the "include/constants_logstream.php" file.

regards,
Andre
User avatar
alorbach
Site Admin
 
Posts: 1127
Joined: Thu Feb 13, 2003 11:55 am

Re: Message Parser --> order of appliance

Postby DocBrown » Mon Nov 24, 2008 12:54 pm

It seems to be that there are additionla configuration steps neccessary.

I tried this example by definining the following:

$dbmapping['log_table'][SYSLOG_MESSAGE] = "message";

But now how to define the table mapping? Is there any specific directive to be used for that purpose?

I also recognised that there is no explicit table mapping defined for systemevents table that is automatically created by setting up rsyslog for usage with mysql?
Furhtermore where are data source changes doen via Admin Center stored?? I found a table sources within my database but nothing happens to this table, regardless of what action I take via Admin Center
DocBrown
Avarage
 
Posts: 18
Joined: Wed Nov 12, 2008 12:58 pm

Re: Message Parser --> order of appliance

Postby alorbach » Mon Nov 24, 2008 4:27 pm

Code: Select all
$dbmapping['log_table'][SYSLOG_MESSAGE] = "message";


This is your table mapping. Name of the mapping is "log_table". The internal phpLogCon FieldID is "SYSLOG_MESSAGE" and your tablefield is "message".
Define the other fields in this way, and you have your mapping automatically.

Yes there is a mapping for the systemevents table used by rsyslog, it is the default configured "monitorware". So by default there is "monitorware" and "syslogng" available as database mapping. As I wrote earlier, take a look into the file src/include/constants_logstream.php, it contains the table mappings definition at the end of the file.

Regarding the sources, they are stored into the sources table, so you were looking in the right spot.

--
regards,
Andre
User avatar
alorbach
Site Admin
 
Posts: 1127
Joined: Thu Feb 13, 2003 11:55 am

Re: Message Parser --> order of appliance

Postby DocBrown » Mon Nov 24, 2008 10:54 pm

Thanks Andre,

but the sources table seems to be somehow strange. I defined three data sources ponting to three different tables. But the sources table only contains one entry that seems to be the one you define at installation time. Could this be?

About the dbmapping, after switching on "debug user" in the data source definition I got the folowing statement generated SELECT , messaage from log_table. Presumably I think I forgot to define a primary key column that the generator is looking for.
After performing that action it was working perfectly.

Regards
Andrej
DocBrown
Avarage
 
Posts: 18
Joined: Wed Nov 12, 2008 12:58 pm

Re: Message Parser --> order of appliance

Postby alorbach » Tue Nov 25, 2008 10:47 am

Ah ok. Yes the primary key is very important, you also need a mapping for the "SYSLOG_UID" field at least, as this is the primary field used for sorting.

best regards,
Andre Lorbach
User avatar
alorbach
Site Admin
 
Posts: 1127
Joined: Thu Feb 13, 2003 11:55 am

Google Ads



Return to Help

Who is online

Users browsing this forum: No registered users and 0 guests

cron