MonitorWare Knowledge Base

rgerhards wrote:

dlang wrote:

dlang wrote:without the -x I can't trigger a failure (I've got a process doing a hup every second and it's not failing)

is there any chance that the name resolution libraries are either
A. not thread safe
or
B. doing something that makes the thread miss the signal from the parent to exit (I would say disabling signals, but I think that you have them disabled anyway. and I don't know how to operate in a threaded environment enough to know what the signal mechanism is)

Actually, both may be (though I had no indication for this up to now). But you already convinced me that a reverse lookup case make sense, so I will not (yet) investigate further. Maintenance of the lookup cache must be synchronized in any was, so I will start with serializing DNS looksups when I implement that. Then, we can remove that serialization and see if it poses any problem. In any case, if there is no strong standard indication that cache lookups be multi-thread enabled, it is probably better to assume they are not (at least in some environments). So the sync primitive should probably engaged in the default case.

some thoughts on a name lookup cache (I specificly do not say DNS becouse you may not be using DNS to get the names, you may be useing /etc/hosts files, yp, NIS+, LDAP, etc depending on how the system is configured)

I don't think that you need to try to deal with expiration of elements from your cache.
trying to figure out what to expire when is a nightmare (that full blown DNS servers frequently don't get right)
isn't possible with some name sources (/etc/hosts)
and to top it off, it doesn't really change much for a syslog server as you are just dealing with local systems.

just having an option to throw away the cache (on restart, file rotation, or some other signal) is probably good enough

I'm not sure that the cache needs to be common across different input modules.
I strongly suspect that usually different input modules are going to receive logs from different sources, so there isn't as much benefit from sharing a cache, and if each input module is running it's own cache you don't have to worry about serializing between different threads.

• Custom written rsyslog.conf?
• Maintenance Contract?
• Installation support?

dlang wrote:I'm not sure that the cache needs to be common across different input modules.
I strongly suspect that usually different input modules are going to receive logs from different sources, so there isn't as much benefit from sharing a cache, and if each input module is running it's own cache you don't have to worry about serializing between different threads.

While I agree to the initial statement and the basic idea, I think it is a bit dangerous if we use multiple caches, one per input. Especially when we do not expire entries, situations may arise where two caches, at the same point in time, have different information about a single object. So we would have an inconsistent view of reality for these two inputs.

I agree that this is probably acceptable, especially given the conditions you mentioned first. It still makes me feel a bit uneasy.

On the performance side, I tend to agree that the extra memory and potentially multiple initial setup of cache information is acceptable in respect to the gain because of spared sync primitives...

Thinking...