MonitorWare Knowledge Base

Hi all,

There is a workaround for get the timestamp in this format?

Thanks for any hint.

• Custom written rsyslog.conf?
• Maintenance Contract?
• Installation support?

I don't believe so. What are you trying to accomplish? Perhaps we can come up with something to accomplish the same goal.

-HKS

I overlooked the thread, sorry. No, you can not do this.

The reason is that generating a *proper* unix timestamp is quite hard to do. Only the OS has the correct information to build a proper timestamp. If you dig into how time is specified, you'll quickly find out why. Think about leap years and leap *seconds*. Yu'll quickly see that things are really weired - the root cause is that good ole' mother earth doesn't rotate as constant as we would like to see it (among others, blame Jupiter's tidal forces... ). So time, as we as humans experience it, is different from what "mathematical correct time" would be.

A solution would be to take the 99.999999% correct approach that most applications use and simply ignore the subtleties. But this can result in a few seconds drift between the so-generated timestamp and the proper one. Also, it would require lookup tables which I think is not something that belongs into rsyslog (plus, it would require "a bit" of effort on my side to obtain them, which is time I don't have. Without the lookup tables, I think we have an instant drift of 38 (or so) seconds towards the official time.

So I would prefer to follow HKS' route first: what is the use case?

Rainer

Just to clarify that I am not joking: have a quick look at:

http://tycho.usno.navy.mil/systime.html

There are more information about the subtleties of time on that site

Rainer

hkspvt wrote:I don't believe so. What are you trying to accomplish? Perhaps we can come up with something to accomplish the same goal.

-HKS

First of all sorry for my bad English.
I try to explain....The logs that i collect from rsyslog must be passed to an external application, but this application accept the log only with unix timestamp (this application has an own template).

The workaround for now is that in my rsyslog.conf i call a bash script that modify the log format of the timestamp in unix time and then forward the modified log to the application. I think that this solution is very bad solution, but for now is the only solution

Bye
denemici

can you post your bash script? I could probably create something that works equally wrong

This is the script (i'm noob on bash programming, but this work)

Code: Select all

#!/bin/sh

#template get from rsyslog.conf

template=$1

#file to write
log_file=/var/log/messages

host=`echo $template | cut -d'|' -f1`
host_ip=`echo $template | cut -d'|' -f2`
facility=`echo $template | cut -d'|' -f3`
priority=`echo $template | cut -d'|' -f4`
tag=`echo $template | cut -d'|' -f5`
timegenerated=`echo $template | cut -d'|' -f6`
timereported=`echo $template | cut -d'|' -f7`
progname=`echo $template | cut -d'|' -f8`
msg=`echo $template | cut -d'|' -f9`

timegenerated_unix=$(date --date="$timegenerated" +%s)
timereported_unix=$(date --date="$timereported" +%s)

#The new string of log
log=$host'|'$host_ip'|'$facility'|'$priority'|'$tag'|'$timegenerated_unix'|'$timereported_unix'|'$progname'|'$msg

#write in append to log_file, or if i want, i call the external application

`echo $log >> $log_file`

exit 0


So you assume that the timestamps are in local time? (I am more than a noob on bash programming, so this may be a funny question ).

oh, and could you post your template from rsyslog.conf?

Hi rgerhards,
This is the template:

Code: Select all

$template mytemplate,"%HOSTNAME%|%fromhost-ip%|%syslogfacility-text%|%syslogpriority%|%syslogtag%|%timegenerated%|%timereported%|%programname%\|%msg%\n"

and this is the action that use:

Code: Select all

*.info;mail.none;authpriv.none;cron.none                ^/path_to_my_script/script.sh;mytemplate

This is only a test, but works

Bye
denemici

I see. You assume that every timestamp is in local time and from the present year. Thus, you (more precisely: bash) calls into the operating system to obtain a unix timestamp that is valid for this timestamp in local time and the present year. The conversion is done under the current locale, that is in respect to the system's daylight time saving settings.

I think it is hard to generalize this, because rsyslog at least needs to abide time zone information into the calculation, at least if we receive a message with a precise timestamp. OK, let's think about that. Such a message should handle the time zone problem, because the ISO timestamp contains an offset to UTC and that should have been properly generated by the sending system (otherwise the timestamp itself is in error, a case which we do not need to handle). So in order to use the OS's timestamp generation function, I need to apply the offset. Unfortunately, this means I may run out of the current day (e.g. if I receive a timestamp of 11:30p UTC and the receiver is in UTC +1), in which case I would need to translate to day+1, 0:30a. That, in turn, means that I may run out of the current month (both back and forth) and that in turn means that I need to abide to leap year calculations (else Feb, 29th will be handled wrong). In theory, a one-second window of uncertainty remains if there is a leap second just at the time I am calculating. This is probably acceptable. So there seems to be a way, but it sounds rather complex to do it (mostly) right.

I'll try to poke around some places to get ideas and if I feel confident this is something that could be implemented in a sufficiently correct way, I'll do it. But I am hesitant to implement something that does not work in a clearly defined set of use cases (that's why this feature is not yet present, btw).

Any pointers and suggestions are welcome.

Rainer