MonitorWare Knowledge Base

I'd like to run rsyslog in an embedded environment, so I'm curious as to rsyslog's memory requirements. How much memory does it require for code and for data? I have been reading and searching the documentation, but didn't find this mentioned.

I'd appreciate any information you could provide.


Thanks,
Don

• Custom written rsyslog.conf?
• Maintenance Contract?
• Installation support?

I've got rsyslogd running on an FC9 host and used "pmap -d" to examine its memory usage. There are three anonymous memory seqments reported, each of size 10MB. I've tried modifying rsyslog.conf, but all three segments are still in use. Here's what I'm currently using for the conf file:

$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat
$ModLoad imuxsock.so
$ModLoad imklog.so
*.* @@w.x.y.z

At a minimum I need to log messages to a remote server via TCP. Is there any way to reduce this 30M of anonymous memory?


Thanks,
Don

I'd say these segments are most likely thread stacks. Or maybe related to dlloading the modules. But my best bet is the stack. So you may tweak it to be lower, 1 to 2 MB should be sufficient (but I've never tested this).

Rainer

Rainer,

Thanks for the information. You were correct - I reduced the default stack size for processes via the ulimit command, and the amount of anonymous memory used by rsyslogd in those 3 large segments was reduced accordingly.


Don

Hi Don, thanks for letting me know. This is useful information. I have added some information to the wiki:

http://wiki.rsyslog.com/index.php/Reducing_memory_usage

Rainer

I ran some simple tests using different stack sizes for rsyslogd, running on an FC9 host. I was able to run it successfully using a stack size of 128K. I had a test process that sent 500K log messages in a tight loop via TCP or RELP to an rsyslog server on another host (running mysql). I ran this test multiple times, with no problems.

So it appears that a relatively small stack size can be used, which is encouraging for embedded projects. I haven't tried anything less than 128K yet.


Don

Don,

thanks for sharing this information. This is very good to know. I'll also keep the stack size on my head when duing further development. In general, I tend to stick with low stack usage and this seems to have worked out well.

Thanks again,
Rainer

Hi,

after noticing the relatively large rsyslogd memory consumption, I wanted to reduce the stack size.
Instead of using the ulimit option (and possibly interfere with other programs), I choose to patch the code instead and use the 'pthread options'.

Ideally this should probably be an opt-in option, either by command-line parameter or configfile parameter.

Patch against stable 4.4.2:

Code: Select all

--- rsyslog-4.4.2/threads.c.orig   2010-01-11 09:20:44.000000000 +0100
+++ rsyslog-4.4.2/threads.c   2010-01-11 09:27:23.000000000 +0100
@@ -37,6 +37,8 @@
#include "linkedlist.h"
#include "threads.h"

+static pthread_attr_t pthreadGlobalAttr;
+
/* linked list of currently-known threads */
static linkedList_t llThrds;

@@ -159,7 +159,11 @@
   pThis->bIsActive = 1;
   pThis->pUsrThrdMain = thrdMain;
   pThis->pAfterRun = afterRun;
+
+   /* Give threads a smaller stack, 512 KiB should be sufficient */
+   pthread_attr_init(&pthreadGlobalAttr);
+   pthread_attr_setstacksize(&pthreadGlobalAttr, 512 * 1024);
-   i = pthread_create(&pThis->thrdID, NULL, thrdStarter, pThis);
+   i = pthread_create(&pThis->thrdID, &pthreadGlobalAttr, thrdStarter, pThis);
   CHKiRet(llAppend(&llThrds, NULL, pThis));

finalize_it:
--- rsyslog-4.4.2/runtime/wtp.c.orig   2010-01-11 10:21:53.000000000 +0100
+++ rsyslog-4.4.2/runtime/wtp.c   2010-01-11 10:24:51.000000000 +0100
@@ -56,6 +56,7 @@
/* static data */
DEFobjStaticHelpers
DEFobjCurrIf(glbl)
+static pthread_attr_t pthreadGlobalAttr;

/* forward-definitions */

@@ -506,7 +506,11 @@

   pWti = pThis->pWrkr[i];
   wtiSetState(pWti, eWRKTHRD_RUN_CREATED, 0, LOCK_MUTEX);
+
+   /* Give threads a smaller stack, 512 KiB should be sufficient */
+   pthread_attr_init(&pthreadGlobalAttr);
+   pthread_attr_setstacksize(&pthreadGlobalAttr, 512 * 1024);
-   iState = pthread_create(&(pWti->thrdID), NULL, wtpWorker, (void*) pWti);
+   iState = pthread_create(&(pWti->thrdID), &pthreadGlobalAttr, wtpWorker, (void*) pWti);
   dbgprintf("%s: started with state %d, num workers now %d\n",
        wtpGetDbgHdr(pThis), iState, pThis->iCurNumWrkThrd);



Hi,

interesting patch, but you are right, this needs to be configurable in order to go into the main branches. But I have to admit I wonder that this is actually causing an issue to you: after all, only address space is allocated, not real memory. Also, a typical system runs on less than 10 threads, so the address space saved is not that much. How many threads do you use on average? I am asking because I wonder if that was really the root cause of what you experienced - a filling-up queue requires considerably more (and real) memory...

Rainer

For me the issue is probably more in the brain and not a 'real' issue. At least not yet, longer term monitoring/usage I have yet to do.

The environment I run rsyslogd in is also intended for 'smaller' boxes, think 64 MiB RAM instead of 8+ GiB.
I monitor memory usage with the help of VmSize from /proc/<pid>/status, and while viewing/monitoring that, rsyslogd stands out like a flagpost (a very big one I might add ).
Decreasing stack size puts rsyslogd below my 'radar'.