rsyslog creates two files instead of one(one have cut name)

General discussions here

Moderator: rgerhards

Google Ads


rsyslog creates two files instead of one(one have cut name)

Postby IgorS » Wed Jun 21, 2017 9:59 am

Hi, i have strange issue where rsyslog is creating two files on disk(instead of one).

For example, my configuration looks like this(sorry for the long names but this is required in order to explain the problem):

template(name="templateA" type="string" string="/opt/destination/files/logs/Wei_BDBvSTAGE_stage_prod_tp__rsyslog-tcp-517-locoA/%FROMHOST-IP%-log_for_Wei_BDBvSTAGE_stage_prod_tp__rsyslog-tcp-517-locaA-Wei_BDBvSTAGE_stage_prod_tp__rsyslog-tcp-517-locoA_on_port_517-siteA-bdb_syslog.log")

# Rule sets
ruleset(name="rule_for_port_517") {
if ($fromhost-ip startswith ["<several IP addresses>"]) then {
?templateA;rawmsgonly
stop
}
?unclassified_data;rawmsgonly
stop
}
# Input bindings
input(type="imtcp" supportOctetCountedFraming="off" port="517" ruleset="rule_for_port_517")


If i send just one message on disk i see the following:

[root@jup ~]# ll /opt/destination/files/logs/Wei_BDBvSTAGE_stage_prod_tp__rsyslog-tcp-517-locoA/
-rw-r--r-- 1 root root 16 Jun 21 11:51 10.10.0.102-log_for_Wei_BDBvSTAGE_stage_prod_tp__rsyslog-tcp-517-locaA-Wei_BDBvSTAGE_stage_prod_tp__rsyslog-tcp-517-locoA
-rw-r--r-- 1 root root 0 Jun 21 11:51 10.10.0.102-log_for_Wei_BDBvSTAGE_stage_prod_tp__rsyslog-tcp-517-locaA-Wei_BDBvSTAGE_stage_prod_tp__rsyslog-tcp-517-locoA_on_port_517-siteA-bdb_syslog.log

Two files has been created, file name where data actually ends, is cut

Is there some size limitation of file name(or file path)?

For example: wc -m OR wc -c on file path returns 234. That shouldn't create problem as far i understand
[root@jup ~]# echo "/opt/destination/files/logs/Wei_BDBvSTAGE_stage_prod_tp__rsyslog-tcp-517-locoA/10.10.0.102-log_for_Wei_BDBvSTAGE_stage_prod_tp__rsyslog-tcp-517-locaA-Wei_BDBvSTAGE_stage_prod_tp__rsyslog-tcp-517-locoA_on_port_517-siteA-bdb_syslog.log" | wc -m
234

In the log i see the following (in bold is line where 'cut' file is being used):

5084.708095499:main Q:Reg/w0 : omfile: file to log to: /opt/destination/files/logs/Wei_BDBvSTAGE_stage_prod_tp__rsyslog-tcp-517-locoA/10.10.0.102-log_for_Wei_BDBvSTAGE_stage_prod_tp__rsyslog-tcp-517-locaA-Wei_BDBvSTAGE_stage_prod_tp__rsyslog-tcp-517-locoA_on_port_517-siteA-bdb_syslog.log
5084.708162509:main Q:Reg/w0 : file stream 10.10.0.102-log_for_Wei_BDBvSTAGE_stage_prod_tp__rsyslog-tcp-517-locaA-Wei_BDBvSTAGE_stage_prod_tp__rsyslog-tcp-517-locoA params: flush interval 0, async write 0
5084.708167423:main Q:Reg/w0 : Added new entry 0 for file cache, file '/opt/destination/files/logs/Wei_BDBvSTAGE_stage_prod_tp__rsyslog-tcp-517-locoA/10.10.0.102-log_for_Wei_BDBvSTAGE_stage_prod_tp__rsyslog-tcp-517-locaA-Wei_BDBvSTAGE_stage_prod_tp__rsyslog-tcp-517-locoA_on_port_517-siteA-bdb_syslog.log'.
5084.708168967:main Q:Reg/w0 : omfile: write to stream, pData->pStrm 0x7fc900003a30, lenBuf 16, strt data #message right#

5084.708171580:main Q:Reg/w0 : strm 0x7fc900003a30: strmFlushinternal: file -1(10.10.0.102-log_for_Wei_BDBvSTAGE_stage_prod_tp__rsyslog-tcp-517-locaA-Wei_BDBvSTAGE_stage_prod_tp__rsyslog-tcp-517-locoA) flush, buflen 16
5084.708173309:main Q:Reg/w0 : strm 0x7fc900003a30: file -1(10.10.0.102-log_for_Wei_BDBvSTAGE_stage_prod_tp__rsyslog-tcp-517-locaA-Wei_BDBvSTAGE_stage_prod_tp__rsyslog-tcp-517-locoA) doWriteInternal: bFlush 1
5084.708174731:main Q:Reg/w0 : strmPhysWrite, stream 0x7fc900003a30, len 16
5084.708189324:main Q:Reg/w0 : file '/opt/destination/files/logs/Wei_BDBvSTAGE_stage_prod_tp__rsyslog-tcp-517-locoA/10.10.0.102-log_for_Wei_BDBvSTAGE_stage_prod_tp__rsyslog-tcp-517-locaA-Wei_BDBvSTAGE_stage_prod_tp__rsyslog-tcp-517-locoA' opened as #15 with mode 420
5084.708193609:main Q:Reg/w0 : strm 0x7fc900003a30: opened file '/opt/destination/files/logs/Wei_BDBvSTAGE_stage_prod_tp__rsyslog-tcp-517-locoA/10.10.0.102-log_for_Wei_BDBvSTAGE_stage_prod_tp__rsyslog-tcp-517-locaA-Wei_BDBvSTAGE_stage_prod_tp__rsyslog-tcp-517-locoA' for WRITE as 15
5084.708201921:main Q:Reg/w0 : strm 0x7fc900003a30: file 15 write wrote 16 bytes
5084.708203694:main Q:Reg/w0 : Action 0 transitioned to state: rdy
IgorS
New
 
Posts: 8
Joined: Thu Mar 16, 2017 10:32 am

Urgent Question?

  • Pulling out your Hair?
  • Wasting Time and Money?
  • Deadline Approaching?

Re: rsyslog creates two files instead of one(one have cut na

Postby IgorS » Thu Jun 22, 2017 10:37 am

I've try to isolate the problem and drop dynafile generation. Snippet from conf file looks like this:

ruleset(name="rule_for_port_517") {
if ($fromhost-ip startswith ["<some IPs>"]) then {
*.* /data/puloadp/rsyslog/Wei_TPPvSTAGE_monitor_stage_prod_tp__rsyslog-tcp-517-locoA/asdfghjklzasdfghjklzasdfghjklzasdfghjklzasdfghjklzasdfghjklzasdfghjklzasdfghjklzasdfghjklzasdfghjklzasdfghjklzasdfghjk
stop
}
?unclassified_data;rawmsgonly
stop
}

This works as long as file path name size doesn't cross 200. When it cross 200 in log i see this message:

4084.349811551:main thread : Called LogMsg, msg: invalid character in selector line - ';template' expected
4084.349898064:main thread : tried selector action for builtin:omfile: -3000
4084.349902461:main thread : error -3000 parsing config line

Btw, its not a problem to create file (manually) on disk with absolute path size bigger then 200.
IgorS
New
 
Posts: 8
Joined: Thu Mar 16, 2017 10:32 am

Google Ads



Return to General

Who is online

Users browsing this forum: No registered users and 1 guest

cron