MonitorWare Knowledge Base

When using the disk file source type, It seems like it is only possible to use a single file as the source. This seems very limited. In particular, for my centralized rsyslog server, I have files separated by host, facility, and date, so monitoring one file would only let me see one day's worth of logs on one host in one facility!

Perhaps my needs would be better met with a database?

Hi,

you can use multiple files. There is a hardcoded limit inside the code, if I correctly remember it is around 200 files. You can always increase it by modifying its #define.

See this doc page for how to add two or more files:

http://www.rsyslog.com/doc-imfile.html

Rainer

Hi,

you can have an unlimited count of sources, so you can define one source for each logfile you want to view within phpLogCon.
For example here I have 2 different sources defined in config.php:

Code: Select all

$CFG['Sources']['Source1']['ID'] = 'Source1';
$CFG['Sources']['Source1']['Name'] = 'Syslog.log';
$CFG['Sources']['Source1']['SourceType'] = 1;
$CFG['Sources']['Source1']['LogLineType'] = 'syslog';
$CFG['Sources']['Source1']['DiskFile'] = '/var/log/syslog';

$CFG['Sources']['Source2']['ID'] = 'Source2';
$CFG['Sources']['Source2']['Name'] = 'Mail.log';
$CFG['Sources']['Source2']['SourceType'] = 1;
$CFG['Sources']['Source2']['LogLineType'] = 'syslog';
$CFG['Sources']['Source2']['DiskFile'] = '/var/log/mail.log';

--
Andre

It is not possible to have multiple files combined into a single source, though, correct?
And if that's possible, is it possible to have multiple files be matched by wildcards?

The desired functionality is to be able to get all messages that are sent to a centralized log server. Maybe it'd be simpler to just add a line to dump all messages into one file in the server configuration, though.

Would having to look at many files cause phplogcon to bog down (especially during complex searches)? I'm currently looking into setting up a database to hold all messages for performance.

Thanks!

From the architect's point of view, supporting multiple files per logstream is not that much of a performance problem (except, of course, if there are many large ones and a search with few results - but that's always the case...). However, there is certain amount of extra logic involved. It becomes extremely complex if the files have like timestamps. So if you would like to view the kernel and mail log in a single source, we would either need to temporarily merge them (impossible due to performance) or display first all kernel and then all mail log messages. This is probably not what you really want.

Support for multiple files is in the spec set, but so far we have decided there are much more pressing issues right now. Also, for the reason outlined above, the idea was to support it so that multiple days (history files) can be supported.

To get different sets with like timestamps, you need to have different source, make the syslogd shuffle everything into a single file, or use the database. There are some limits with what you can file sources make do and sequence is the most limiting factor.

I hope this clarifies,
Rainer

I think in this case it would make sense to got for the database environment.
Rainer already said it, we would need to merge the files on runtime which is quit difficult and would cost a lot of performance.

best regards,
Andre Lorbach