MonitorWare Knowledge Base

I have 2 or 3 systems pointing to the appliance and they log pretty lightly. But it seems if I filter more than one system (IE. exclude it from the results) it pegs apache at 100% cpu and most times, times out. I can increase the timeout but it still has performance issues. I have the vmware package installed on a server with 2 quad cores and 8 gigs of ram. Memory does not seem to be the issue so I was wondering about assigning 2 cores to the appliance but it seems that the kernel was not compiled for SMP.

Is there another reason for the performance problems and if not can the next version support SMP?

Thanks.

Hy Pyke

I would be in the same boat as Hypyke with the exact same system (2 quad cores, 8GB of RAM) and i am having the same issues - except the logging is much more intense. I have a few mail gateway servers shooting logs off to the server plus dns and web servers so it's getting hit a lot, and whenever i try to search or seemingly at random times apache2 gets pegged @ 100%CPU Usage - which is frustrating as the page always times out.
SMP Kernel Support would be nice, and perhaps a more efficient http daemon (lighttpd IMO)

Sorry, I did overlook the initial post. SMP support is a good idea, but I would first like to isolate the root cause of the performance issue. It looks like this is somewhere in phpLogCon, so I will talk to Andre, the lead developer there.

Did you increase the memory assigned to the VM? I kept it low, maybe too low. Even if it is too low, it should not account for 100% CPU, but maybe it is worth a try.

Many thanks for your feedback, I will keep you posted.

Rainer

I am currently reproducing this problem with some filters, and analysing the process.
I see the apache service assuming lots of CPU, and something is going wrong the SQL queries.

This seems to be a filtering bug in phplogcon introduced in one of the last versions. I am looking into this right now.

--
regards,
Andre

I found and fixed the bug in phplogcon, to fix it manually please edit the file "classes/logstreamdb.php"

Search for this code:

Code: Select all

      // Copy rows into the buffer!
      $iBegin = $this->_currentRecordNum;
      while ($myRow = mysql_fetch_array($myquery,  MYSQL_ASSOC))
      {
         $this->bufferedRecords[$iBegin] = $myRow;
         $iBegin++;
      }


Directly after, add this code:

Code: Select all

      // --- Check if results were found
      if ( $iBegin == $this->_currentRecordNum )
         return ERROR_NOMORERECORDS;
      // ---


This bug will be fixed in the next phpLogCon release (most likely this will be v2.5.20)

best regards,
Andre Lorbach

I'll update the appliance as soon as some additonal testing is conducated. Feedback would be appreciated.

Rainer

Little update, fixed the issue and release phpLogCon 2.5.20, download here:
http://www.phplogcon.org/Downloads-req- ... d-47.phtml

You can manually update your syslogappliance installations with this release, just copy all files over your existing installation in /var/www/

best regards,
Andre Lorbach

Thanks Guys!
That little piece of code seems to have fixed my issue! I had in fact increased the RAM to 512MB just for fun but the issue was persistent.
As soon as i added that snippet of code i can now login (i couldnt even login before without it timing out) and searching is actually working.
Thanks for the fast response - much appreciated!
*edit*
As an edit, my CPU Load Averages are still pretty high, and when searching for a string with the page refresh enabled it does time out when trying to refresh - but at least it works i guess. My script timeout is set to 45 seconds yet it still times out, just an FYI.

And as an afterthought - phplogcon still times out when im on the default view trying to refresh @ 60 second intervals, and CPU is still pegged at 100 percent when this is happening (apache again)

When it doesn't timeout, can you tell me how many "DB queries" it shows at the bottom of the page in phplogcon?
Edit: Another question, do you have the option "Enable Row Counting" in your logstream source configuration enabled? This option can cause performance problems on larger databases

When it does not time out (IE when it is normally working) it uses 11 DB Queries.
I am using the Appliance Version and Enable Row Counting is set to 'No'.
I can definitely re-create the issue when i attempt to use the search function. When i attempt to filter by anything, Apache goes to 100% and the script times out.
It has stopped happening (quite as often) when sitting on the index page watching the logs come in, but it still happens almost every time that i search or filter.
Not sure how many queries it attempts to open when it times out as i cant see the number being displayed.

Just to make sure, have you updated your phplogcon installation to Version 2.5.20 already?
I have about 500000 records now in my vmware syslogappliance, and not expiriencing problems when using the filters - yet.

Andre,

is the full update required or just the fix you posted?

Rainer

I fixed this in 2 logstream classes, logstreamdb and logstreampdo, so I recommend to use the full update.

regards,
Andre Lorbach

Ahh - i was using just the fix that you posted above.
Will replace with the new version and let you know if i run into any more issues.
Thanks!