Problems queueing when logging around 800 msgs/second

General discussions here

Moderator: rgerhards

Problems queueing when logging around 800 msgs/second

Postby vmix on Fri Jun 20, 2008 7:53 pm

I didn't see any previous reports on the forums about this, so here's the problem:

When logging about 800 messages per second to remote host and causing a service interruption via an rsyslog restart (with 5 second pause) on the remote host, I see that some messages from 2-3 seconds into the service interruption are lost, while all of them from 4-5 seconds into the service interruption are lost. Also, when doubling the messages per second, it appears that the queueing does not work as soon as the service interruption begins. Testing with 500 messages per second works fine, and sending 500 from three hosts at a time, for a total of 1,500 messages per second works fine, so it appears to be a client limitation of sorts. There is never a problem resuming after the interruption. Any thoughts?

Dan.
User avatar
vmix
Advanced
 
Posts: 32
Joined: Tue May 06, 2008 5:48 pm
Location: San Diego, CA

Professional Services Information

  • Custom written rsyslog.conf?
  • Maintenance Contract?
  • Installation support?

Re: Problems queueing when logging around 800 msgs/second

Postby vmix on Fri Jun 20, 2008 7:54 pm

I had forget to mention that we are using the latest available stable release, which is rsyslog v3.16.1 on all hosts. Thanks for any input.

Dan.
User avatar
vmix
Advanced
 
Posts: 32
Joined: Tue May 06, 2008 5:48 pm
Location: San Diego, CA

Re: Problems queueing when logging around 800 msgs/second

Postby rgerhards on Fri Jun 20, 2008 8:15 pm

Hi Dan,

I assume you use plain tcp syslog. This is not a totally reliable protocol. Message loss, as you describe, is expected. See here for reasoning:

http://blog.gerhards.net/2008/04/on-unr ... yslog.html

The simple cure is to use RELP (RELP was introduced to overcome this limitation).

Rainer
User avatar
rgerhards
Site Admin
 
Posts: 1662
Joined: Thu Feb 13, 2003 11:57 am

Google Ads



Return to General

Who is online

Users browsing this forum: No registered users and 0 guests

cron