1,000,000 80-character messages/sec with Rsyslog 5.6.3

General discussions here

Moderator: rgerhards

Google Ads


1,000,000 80-character messages/sec with Rsyslog 5.6.3

Postby kechambe » Wed Feb 23, 2011 3:00 pm

My team recently investigated Rsyslog as a possible solution for centralized logging. Syslog has traditionally been a performance dog and there was early concern that Rsyslog might be too.

I'm very pleased to report that it took just 784 seconds to send a total of 800,000,000 x 80 character messages from 8 x Rsyslog 5.6.3 sender VMs to 1 x Rsyslog 5.6.3 receiver VM. This works out to 1.02 million messages/sec!!

Hats off to Rainer and others that have contributed to Rsyslog. :-)

Rsyslog 5.6.3 Receiver VM Stats Summary:

* 1.02 million 80-character messages/sec over TCP
* 1.39 Gbps (reported by ESX)
* 70% CPU (reported by ESX)

Test topology:

100,000,000 x 80-character log ---> Rsyslog 5.6.3 Sender VM ---v
100,000,000 x 80-character log ---> Rsyslog 5.6.3 Sender VM ---v
100,000,000 x 80-character log ---> Rsyslog 5.6.3 Sender VM ---v
100,000,000 x 80-character log ---> Rsyslog 5.6.3 Sender VM ---v
100,000,000 x 80-character log ---> Rsyslog 5.6.3 Sender VM ---v
100,000,000 x 80-character log ---> Rsyslog 5.6.3 Sender VM ---v
100,000,000 x 80-character log ---> Rsyslog 5.6.3 Sender VM ---v
100,000,000 x 80-character log ---> Rsyslog 5.6.3 Sender VM ---v

TCP ---> 10Gig Ethernet ---> Rsyslog 5.6.3 Receiver ---> /dev/null

Sender VM Details:

* Rsyslog 5.6.3
* CentOS 5.5
* VM config = 2-vCPU / 8GB RAM / VMXNET3 vNIC
* Host config = VMware ESX 4.1 / Cisco UCS C200 w/2 Intel X5670 2.93GHz 6-core CPUs / 10Gb CNA

Receiver VM Details:

* Rsyslog 5.6.3
* CentOS 5.5
* VM config = 4-vCPU / 8GB RAM / VMXNET3 vNIC
* Host config = VMware ESX 4.1 / Cisco UCS B200 w/ 2 x Intel X5570 2.93GHz 4-core CPUs / 10Gb CNA
* /etc/rsyslog.conf non-default changes:

#### GLOBAL DIRECTIVES ####
$MainMsgQueueWorkerThreads 2

#### RULES ####
*.* /dev/null
kechambe
New
 
Posts: 2
Joined: Wed Feb 23, 2011 2:17 pm

Urgent Question?

  • Pulling out your Hair?
  • Wasting Time and Money?
  • Deadline Approaching?

Re: 1,000,000 80-character messages/sec with Rsyslog 5.6.3

Postby rgerhards » Wed Feb 23, 2011 3:12 pm

Thanks for sharing the good news. Question: have you used imtcp or imptcp (note the "p"). The latter should provide around 7% more performance...

Rainer
rgerhards
Site Admin
 
Posts: 3792
Joined: Thu Feb 13, 2003 11:57 am

Google Ads



Return to General

Who is online

Users browsing this forum: No registered users and 0 guests

cron