Dynamic filenames feature bug!

Diskussions related to the development of PhpLogCon

Google Ads


Dynamic filenames feature bug!

Postby sire » Thu Nov 12, 2009 1:32 pm

The "Dynamic filenames" feature ignores log files, whose date is older than current date.
For example, on my log server I have the following directory structure:
Code: Select all
/var/log/remote/%HOSTNAME%/%$YEAR%/%$YEAR%-%$MONTH%/%$YEAR%-%$MONTH%-%$DAY%/auth.log
/var/log/remote/%HOSTNAME%/%$YEAR%/%$YEAR%-%$MONTH%/%$YEAR%-%$MONTH%-%$DAY%/daemon.log
/var/log/remote/%HOSTNAME%/%$YEAR%/%$YEAR%-%$MONTH%/%$YEAR%-%$MONTH%-%$DAY%/mail.log
...

When I try to look through yesterday's logs by turning over pages in phpLogCon, the logs end at the beginning of the current day. Searching with dates other than current date also fails.

I think that happens because phpLogCon substitutes date replacement characters with today's date once at startup, while it should do this every time a user requests logs according to the time period requested.

Can anyone fix it, please?
sire
Avarage
 
Posts: 18
Joined: Thu Nov 12, 2009 1:19 pm

Urgent Question?

  • Pulling out your Hair?
  • Wasting Time and Money?
  • Deadline Approaching?

Re: Dynamic filenames feature bug!

Postby sire » Thu Nov 12, 2009 4:33 pm

And it would be even greater if phpLogCon supported %HOSTNAME% as well as date replacement characters.
sire
Avarage
 
Posts: 18
Joined: Thu Nov 12, 2009 1:19 pm

Re: Dynamic filenames feature bug!

Postby alorbach » Fri Nov 13, 2009 10:29 am

Hi,

supporting the other replacement characters shouldn't be a problem.
But regarding the other issue, this is not really a bug, this is how the dynamic filenames feature is supposed to work.
I understand the need for you to be able to read the yesterdays logfile, but this is currently not possible.

best regards,
Andre Lorbach
alorbach
Site Admin
 
Posts: 1627
Joined: Thu Feb 13, 2003 11:55 am

Re: Dynamic filenames feature bug!

Postby sire » Fri Nov 13, 2009 10:58 am

alorbach wrote:Hi,
supporting the other replacement characters shouldn't be a problem.
But regarding the other issue, this is not really a bug, this is how the dynamic filenames feature is supposed to work.
I understand the need for you to be able to read the yesterdays logfile, but this is currently not possible.


Hi Andre,

After writing my message I have read in an other thread of the forum about performance issues that would arise in case of linking log files on the fly. Ok, if this is practically impossible, lets give a user an ability to select a file he needs. In case if files are sorted as in my first post (or smth. like that), it is very easy to select a date desired. What do you think?
sire
Avarage
 
Posts: 18
Joined: Thu Nov 12, 2009 1:19 pm

Re: Dynamic filenames feature bug!

Postby alorbach » Fri Nov 13, 2009 12:08 pm

The problem is this wouldn't be possible to fit into how the logstream work in phpLogCon very well.
We may could go the way awstats works, which allows you to configure dynamic file names like this:
/var/log/remote/%HOSTNAME%/%$YEAR%/%$YEAR%-%$MONTH%/%$YEAR%-%$MONTH%-%$DAY-1%/auth.log

Which means it would be one day less, so the yesterdays logfile. You would need to configure two sources, one for today and one for yesterdays logfiles.

best regards,
Andre Lorbach
alorbach
Site Admin
 
Posts: 1627
Joined: Thu Feb 13, 2003 11:55 am

Re: Dynamic filenames feature bug!

Postby sire » Fri Nov 13, 2009 12:41 pm

Just today and yesterday may be not enough. Why not implement a date chooser?
sire
Avarage
 
Posts: 18
Joined: Thu Nov 12, 2009 1:19 pm

Re: Dynamic filenames feature bug!

Postby rgerhards » Fri Nov 13, 2009 1:59 pm

I, too, would find this useful. But it is a lot of work, and it looks like there currently is simply no time to do that. Contributions are of course always happily accepted ;)

Rainer
rgerhards
Site Admin
 
Posts: 3806
Joined: Thu Feb 13, 2003 11:57 am

Re: Dynamic filenames feature bug!

Postby sire » Fri Nov 13, 2009 3:54 pm

rgerhards wrote:I, too, would find this useful. But it is a lot of work, and it looks like there currently is simply no time to do that. Contributions are of course always happily accepted ;)
Rainer

I have just made some changes in phpLogCon to make it parse %H replacement character as a hostname substitution in log file path. I have introduced a new config.php option:
Code: Select all
$CFG['Sources']['Source12']['Hostnames']= 'host1 host2 host3'

that specifies a space-separated list of hostnames for a log file specified as
Code: Select all
$CFG['Sources']['Source12']['DiskFile']= '/var/log/remote/%H/%Y/%Y-%m/%Y-%m-%d/auth.log'

The patch is attached to this message.
Attachments
phplogcon-2.7.2-hostname.patch.gz
The patch file
(988 Bytes) Downloaded 277 times
sire
Avarage
 
Posts: 18
Joined: Thu Nov 12, 2009 1:19 pm

Re: Dynamic filenames feature bug!

Postby sire » Fri Nov 13, 2009 5:26 pm

sire wrote:
rgerhards wrote:I, too, would find this useful. But it is a lot of work, and it looks like there currently is simply no time to do that. Contributions are of course always happily accepted ;)
Rainer

I have just made some changes in phpLogCon to make it parse %H replacement character as a hostname substitution in log file path. I have introduced a new config.php option:
Code: Select all
$CFG['Sources']['Source12']['Hostnames']= 'host1 host2 host3'

that specifies a space-separated list of hostnames for a log file specified as
Code: Select all
$CFG['Sources']['Source12']['DiskFile']= '/var/log/remote/%H/%Y/%Y-%m/%Y-%m-%d/auth.log'

The patch is attached to this message.

I have slightly reworked the patch. The patch v2 now supports one more option:
Code: Select all
$CFG['SkipUnexistingLogFiles'] = 1;

Which if set to 1 doesn't show log files that don't exist on disk.
Attachments
phplogcon-2.7.2-hostname-v2.patch.gz
Patch v2
(1.64 KiB) Downloaded 281 times
sire
Avarage
 
Posts: 18
Joined: Thu Nov 12, 2009 1:19 pm

Re: Dynamic filenames feature bug!

Postby sire » Mon Nov 16, 2009 1:41 pm

Hi all,

I have managed to make phpLogCon to open older log files when a user specifies date interval in a search filter. A patch attached to this message introduces the mentioned functionality together with the hostname %H replacement character support in my previous patch.

This is a result of several hours of work, it is not the best way to implement the mentioned features. In my opinion there should be a date chooser, probably looking like a calendar, and a hostname selector drop-down list box situated at the top of the phpLogCon's main page.

However AFAIK this is the only way to look through old log files existing at this moment.

Best Regards,
Sergey Sireskin
Attachments
phplogcon-2.7.2-dynamic.patch.gz
Improvements in dynamic file parsing patch
(3.64 KiB) Downloaded 287 times
sire
Avarage
 
Posts: 18
Joined: Thu Nov 12, 2009 1:19 pm

Re: Dynamic filenames feature bug!

Postby sire » Wed Nov 18, 2009 10:38 am

Hi all,

I have slightly changed my dynamic files patch to fix php warnings.
The forum engine doesn't allow to modify messages, so I'll post my modified dynamic patch in this new message.

Best Regards,
Sergey Sireskin
Attachments
phplogcon-2.7.2-dynamic.patch.gz
New dynamic files patch
(3.62 KiB) Downloaded 276 times
sire
Avarage
 
Posts: 18
Joined: Thu Nov 12, 2009 1:19 pm

Re: Dynamic filenames feature bug!

Postby sire » Thu Dec 03, 2009 4:13 pm

Hi All,

Here is a new version of my dynamic files patch that is significantly faster than all of the previous as it uses built-in php function glob() instead of executing external ls command.

Best Regards,
Sergey Sireskin
Attachments
phplogcon-2.7.3-dynamic.patch.gz
(3.62 KiB) Downloaded 306 times
sire
Avarage
 
Posts: 18
Joined: Thu Nov 12, 2009 1:19 pm

Google Ads



Return to Developer's Corner

Who is online

Users browsing this forum: No registered users and 0 guests

cron