rsyslog can't send old data

Everything related with getting rsyslog up and running (but not beyond that point ;))

Moderator: rgerhards

Google Ads


rsyslog can't send old data

Postby protoss5945 » Fri May 19, 2017 10:04 am

I use rsyslogd 8.23.0 on centos6.5 and send log to my local flume
now I have a problem
rsyslog can't send old data through I had set action.resumeRetryCount="-1"
my conf:
global (
maxMessageSize="150m"
workDirectory="/var/lib/rsyslog"
preserveFQDN="on"
action.reportSuspension="on"
)

main_queue (
queue.filename="mainQ"
queue.type="linkedlist"
queue.size="400000"
queue.timeoutenqueue="1000"
queue.maxfilesize="512M"
queue.maxdiskspace="8G"
queue.dequeuebatchsize="2000"
queue.workerthreads="4"
queue.workerthreadminimummessages="100000"
queue.discardseverity="3"
queue.checkpointinterval="10"
queue.timeoutshutdown="10"
queue.timeoutactioncompletion="10"
queue.timeoutworkerthreadshutdown="10"
queue.saveonshutdown="on"
)
module(load="imuxsock") # provides support for local system logging (e.g. via logger command)
module(load="imklog") # provides kernel logging support (previously done by rklogd)
module(load="imfile")
$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat
$IncludeConfig /etc/rsyslog.d/*.conf

#cat /etc/rsyslog.d/messagelog.conf
template(name="MessagelogTemplate" type="string" string="<%PRI%>1 %TIMESTAMP:::date-rfc3339% %HOSTNAME% %HOSTNAME% %$year%/%$month%/%$day% %TIMESTAMP:8:15% %msg:::drop-last-lf%\n")

#your inpurt here
input(
type="imfile"
File="/var/log/messages"
Tag="messagelog"
Severity="info"
persiststateinterval="10"
deleteStateOnFileDelete="off"
freshStartTail="on"
reopenOnTruncate="on"
Facility="local7"
)

input(
type="imfile"
File="/var/log/useraudit.log"
Tag="messagelog-useraudit"
Severity="info"
persiststateinterval="10"
deleteStateOnFileDelete="off"
freshStartTail="on"
reopenOnTruncate="on"
Facility="local7"
)

input(
type="imfile"
File="/var/log/secure"
Tag="messagelog-secure"
Severity="info"
persiststateinterval="10"
deleteStateOnFileDelete="off"
freshStartTail="on"
reopenOnTruncate="on"
Facility="local7"
)
#your Ruleset
Ruleset( name="fwd_message_log" )
{
action(
name="fwd_message_log"
type="omfwd"
Target="127.0.0.1"
Port="7142"
Protocol="tcp"
RebindInterval="5000"
action.resumeRetryCount="-1"
template="MessagelogTemplate"
queue.filename="action_fwd_message_log"
queue.size="1500"
queue.dequeuebatchsize="50"
queue.maxdiskspace="1G"
queue.type="linkedlist"
queue.maxfilesize="50M"
queue.saveonshutdown="on"
)
}
#use Ruleset
if ( $syslogfacility-text == "local7" and $programname startswith "message" ) then{
call fwd_message_log
stop

# cd /var/lib/rsyslog
# ll -a
-rw------- 1 root root 12394407 May 16 02:08 action_fwd_message_log.00000019
-rw------- 1 root root 3683881 May 17 22:15 action_fwd_message_log.00000056
-rw------- 1 root root 49830669 May 10 00:05 action_fwd_message_log.00000091
-rw------- 1 root root 14082248 May 8 16:56 action_fwd_message_log.00000108
-rw------- 1 root root 2989412 May 16 08:16 action_fwd_tengine_log.00000004
-rw------- 1 root root 12003300 May 17 17:34 action_fwd_tengine_log.00000017
-rw------- 1 root root 269 May 19 17:01 imfile-state:-usr-local-mallard-pingClient-applog-pingData.log
-rw------- 1 root root 249 May 19 17:01 imfile-state:-usr-local-purge-logs-que.log
-rw------- 1 root root 248 May 19 17:00 imfile-state:-usr-local-sanma-log-access.log
-rw------- 1 root root 256 May 19 16:55 imfile-state:-usr-local-squid-var-logs-error.log
-rw------- 1 root root 253 May 19 17:01 imfile-state:-usr-local-tengine-logs-error.log
-rw------- 1 root root 236 May 19 17:01 imfile-state:-var-log-messages
-rw------- 1 root root 233 May 19 16:59 imfile-state:-var-log-secure
-rw------- 1 root root 240 May 19 16:00 imfile-state:-var-log-useraudit.log
-rw------- 1 root root 6291315 May 19 17:01 stats
-rw------- 1 root root 27347160 May 13 03:26 stats-20170513
-rw------- 1 root root 47024108 May 14 03:20 stats-20170514
-rw------- 1 root root 86014587 May 15 03:12 stats-20170515
-rw------- 1 root root 77319155 May 16 03:35 stats-20170516
-rw------- 1 root root 41911547 May 17 03:50 stats-20170517
-rw------- 1 root root 33229067 May 18 03:14 stats-20170518
-rw------- 1 root root 11619709 May 19 03:45 stats-20170519

the old log can' send to flume(now is 2017/5/19) while the lastest log is working well
problem:
1、how it happend?
2、how to do with it (i can't lose any data)
thanks all
protoss5945
Avarage
 
Posts: 14
Joined: Fri May 19, 2017 9:41 am

Urgent Question?

  • Pulling out your Hair?
  • Wasting Time and Money?
  • Deadline Approaching?

Re: rsyslog can't send old data

Postby protoss5945 » Fri May 19, 2017 10:07 am

i restart the rsyslogd but the log still be there never be sent
protoss5945
Avarage
 
Posts: 14
Joined: Fri May 19, 2017 9:41 am

Re: rsyslog can't send old data

Postby dlang » Sat May 20, 2017 7:21 am

It is not clear to me what isn't working.

What do you expect to be happening, and what is actually happening instead?
dlang
Frequent Poster
 
Posts: 1001
Joined: Mon Sep 15, 2008 7:44 am

Re: rsyslog can't send old data

Postby protoss5945 » Mon May 22, 2017 3:32 am

at first when I found the old data don't be sent i restart the rsyslog then the old data sent
, but now this way doesn't work anymore . and anything else i haven't change . what information do you need
protoss5945
Avarage
 
Posts: 14
Joined: Fri May 19, 2017 9:41 am

Re: rsyslog can't send old data

Postby protoss5945 » Mon May 22, 2017 8:53 am

and i run
/sbin/rsyslogd -n
there is no message
so strange
protoss5945
Avarage
 
Posts: 14
Joined: Fri May 19, 2017 9:41 am

Re: rsyslog can't send old data

Postby protoss5945 » Mon May 22, 2017 9:52 am

the old data size
[root@ rsyslog]# ll -a
total 8038016
drwx------. 2 root root 4096 May 19 20:58 .
drwxr-xr-x. 20 root root 4096 Jun 29 2016 ..
-rw------- 1 root root 50000175 May 12 15:04 action_fwd_puffer_log.00000007
-rw------- 1 root root 50000280 May 13 00:54 action_fwd_puffer_log.00000008
-rw------- 1 root root 50000023 May 13 10:49 action_fwd_puffer_log.00000009
-rw------- 1 root root 50000420 May 13 20:42 action_fwd_puffer_log.00000010
-rw------- 1 root root 50000045 May 14 06:33 action_fwd_puffer_log.00000011
-rw------- 1 root root 50000070 May 14 16:23 action_fwd_puffer_log.00000012
-rw------- 1 root root 50000419 May 15 02:13 action_fwd_puffer_log.00000013
-rw------- 1 root root 50000245 May 15 12:08 action_fwd_puffer_log.00000014
-rw------- 1 root root 18633171 May 15 15:43 action_fwd_puffer_log.00000015
protoss5945
Avarage
 
Posts: 14
Joined: Fri May 19, 2017 9:41 am

Re: rsyslog can't send old data

Postby protoss5945 » Mon May 22, 2017 9:53 am

i want to send all the data but now some old data seems to never be sent
protoss5945
Avarage
 
Posts: 14
Joined: Fri May 19, 2017 9:41 am

Re: rsyslog can't send old data

Postby rgerhards » Mon May 22, 2017 10:02 am

Well, imfile monitors those files that you told it to monitor. It does not know about any of the old files, at least I have not seen any reference to them in the posted config.

For me, I wonder why you don't directly send the log messages when they are generated, instead of picking them up from the files. That would be more robust in any case.

HTH
Rainer
rgerhards
Site Admin
 
Posts: 3801
Joined: Thu Feb 13, 2003 11:57 am

Re: rsyslog can't send old data

Postby protoss5945 » Tue May 23, 2017 4:02 am

i use rsyslog to send kinds of logs like sys and server to flume ,and their configs are almost same except their dir and file name.
ok we onlig see the message log is fllowing

[root@yd-sichuan-chengdu-3-117-139-22-71 rsyslog]# ll -a
total 136920
drwx------. 2 root root 4096 May 23 03:49 .
drwxr-xr-x. 20 root root 4096 Jun 29 2016 ..
-rw------- 1 root root 3683881 May 17 22:15 action_fwd_message_log.00000056

the log is always there
first i restart the rsyslog the log like this can be sent but now this way doesn't work
and i am sure after the rsyslog restart rsyslog had read the old logs ,but i don't know load it to main queue or action queue .and the rsyslog' retry is begin at main queue or action queue? why it miss this log
protoss5945
Avarage
 
Posts: 14
Joined: Fri May 19, 2017 9:41 am

Re: rsyslog can't send old data

Postby dlang » Tue May 23, 2017 10:15 am

what do you mean by 'sent old logs'

do you mean older logs in the same file? or do you mean logs in other files?
dlang
Frequent Poster
 
Posts: 1001
Joined: Mon Sep 15, 2008 7:44 am

Re: rsyslog can't send old data

Postby protoss5945 » Tue May 23, 2017 11:10 am

usually rsyslog send log to flume ,but if something happend like rsyslog shutdown ,rsyslog will dump the data in memory to the disk, then rsyslog become good ,rsyslog will reload the data on disk to memory .and put this data to action queue then send them as usual.
am i right on how rsyslog works? the data on disk which not send far from now ,i call it old log.

the rsyslog wil send logs every minutes. for example:
the messages of the whole day on May 17 ,are sent to flume .but i found some data left on disk .I write a shell which found if there is some data on disk,it woll restart the rsyslog in order to send the data on disk immediately. at first this way gose well but now I found whatever i do with rsyslog ,the data on disk which far away form now never be sent again.

and in my work dir the on_disk_data like flowing be there all the time, looks like never to send.
May 17 22:15 action_fwd_message_log.00000056 (now: 2017/5/23)

if the old log is too much the rsyslog will not start because rsyslog' memory is lee and when i turn it large enough the rsyslog will start,this prove that rsyslog load the old on_disk_data to memory .the i search in debug mode i haven't see how the rsyslog put the data in memory into action queue or retry the on_disk_data.

do i need set some parameter ? or do sth to fix it
protoss5945
Avarage
 
Posts: 14
Joined: Fri May 19, 2017 9:41 am

Re: rsyslog can't send old data

Postby dlang » Tue May 23, 2017 11:23 am

Ok, so you are talking about messages that have been read, but are sitting in a queue and not sent.

The first thing we need to do is to get the impstats data on the disk queues so that we can see what happens. Rsyslog will not delete the last queue file (except when it shuts down) because it can't know that there isn't a message that needs to be added to the file in the near future.

Rsyslog continues to send new messages in real-time, and fills in with old logs as it has time. It is less efficient at pulling and sending the old logs than we would like, but they do get sent.

So we need to see the DA lines from the impstats output to see what is happening with those queues.
dlang
Frequent Poster
 
Posts: 1001
Joined: Mon Sep 15, 2008 7:44 am

Re: rsyslog can't send old data

Postby protoss5945 » Tue May 23, 2017 11:56 am

you mean this?
the lasteset da line:
Tue May 23 18:47:07 2017: { "name": "fwd_message_log queue[DA]", "origin": "core.queue", "size": 0, "enqueued": 0, "full": 0, "discarded.full": 0, "discarded.nf": 0, "maxqsize": 0 }
Tue May 23 18:47:07 2017: { "name": "fwd_message_log queue", "origin": "core.queue", "size": 0, "enqueued": 138, "full": 0, "discarded.full": 0, "discarded.nf": 0, "maxqsize": 11 }
Tue May 23 18:48:07 2017: { "name": "fwd_message_log", "origin": "core.action", "processed": 141, "failed": 0, "suspended": 0, "suspended.duration": 0, "resumed": 0 }
Tue May 23 18:48:07 2017: { "name": "fwd_message_log queue[DA]", "origin": "core.queue", "size": 0, "enqueued": 0, "full": 0, "discarded.full": 0, "discarded.nf": 0, "maxqsize": 0 }
Tue May 23 18:48:07 2017: { "name": "fwd_message_log queue", "origin": "core.queue", "size": 0, "enqueued": 141, "full": 0, "discarded.full": 0, "discarded.nf": 0, "maxqsize": 11 }
Tue May 23 18:49:07 2017: { "name": "fwd_message_log", "origin": "core.action", "processed": 138, "failed": 0, "suspended": 0, "suspended.duration": 0, "resumed": 0 }
Tue May 23 18:49:07 2017: { "name": "fwd_message_log queue[DA]", "origin": "core.queue", "size": 0, "enqueued": 0, "full": 0, "discarded.full": 0, "discarded.nf": 0, "maxqsize": 0 }
Tue May 23 18:49:07 2017: { "name": "fwd_message_log queue", "origin": "core.queue", "size": 0, "enqueued": 138, "full": 0, "discarded.full": 0, "discarded.nf": 0, "maxqsize": 11 }
Tue May 23 18:50:07 2017: { "name": "fwd_message_log", "origin": "core.action", "processed": 138, "failed": 0, "suspended": 0, "suspended.duration": 0, "resumed": 0 }
Tue May 23 18:50:07 2017: { "name": "fwd_message_log queue[DA]", "origin": "core.queue", "size": 0, "enqueued": 0, "full": 0, "discarded.full": 0, "discarded.nf": 0, "maxqsize": 0 }
Tue May 23 18:50:07 2017: { "name": "fwd_message_log queue", "origin": "core.queue", "size": 0, "enqueued": 138, "full": 0, "discarded.full": 0, "discarded.nf": 0, "maxqsize": 11 }

and the ptoblem time da line
Wed May 17 22:15:05 2017: { "name": "global", "origin": "dynstats", "values": { } }
Wed May 17 22:15:05 2017: { "name": "imuxsock", "origin": "imuxsock", "submitted": 36, "ratelimit.discarded": 0, "ratelimit.numratelimiters": 0 }
Wed May 17 22:15:05 2017: { "name": "fwd_squid_log", "origin": "core.action", "processed": 0, "failed": 0, "suspended": 0, "suspended.duration": 0, "resumed": 0 }
Wed May 17 22:15:05 2017: { "name": "fwd_message_log", "origin": "core.action", "processed": 1028, "failed": 0, "suspended": 0, "suspended.duration": 0, "resumed": 0 }
Wed May 17 22:15:05 2017: { "name": "fwd_ping_log", "origin": "core.action", "processed": 20, "failed": 0, "suspended": 0, "suspended.duration": 0, "resumed": 0 }
Wed May 17 22:15:05 2017: { "name": "dynafile cache pingsocket", "origin": "omfile", "requests": 0, "level0": 0, "missed": 0, "evicted": 0, "maxused": 0, "closetimeouts": 0 }
Wed May 17 22:15:05 2017: { "name": "save_ping", "origin": "core.action", "processed": 0, "failed": 0, "suspended": 0, "suspended.duration": 0, "resumed": 0 }
Wed May 17 22:15:05 2017: { "name": "dynafile cache quesocket", "origin": "omfile", "requests": 0, "level0": 0, "missed": 0, "evicted": 0, "maxused": 0, "closetimeouts": 0 }
Wed May 17 22:15:05 2017: { "name": "save_que", "origin": "core.action", "processed": 0, "failed": 0, "suspended": 0, "suspended.duration": 0, "resumed": 0 }
Wed May 17 22:15:05 2017: { "name": "fwd_que_log", "origin": "core.action", "processed": 127, "failed": 0, "suspended": 0, "suspended.duration": 0, "resumed": 0 }
Wed May 17 22:15:05 2017: { "name": "dynafile cache sanmasocket", "origin": "omfile", "requests": 0, "level0": 0, "missed": 0, "evicted": 0, "maxused": 0, "closetimeouts": 0 }
Wed May 17 22:15:05 2017: { "name": "save_sanma", "origin": "core.action", "processed": 0, "failed": 0, "suspended": 0, "suspended.duration": 0, "resumed": 0 }
Wed May 17 22:15:05 2017: { "name": "fwd_sanma_log", "origin": "core.action", "processed": 40, "failed": 0, "suspended": 0, "suspended.duration": 0, "resumed": 0 }
Wed May 17 22:15:05 2017: { "name": "dynafile cache socket", "origin": "omfile", "requests": 0, "level0": 0, "missed": 0, "evicted": 0, "maxused": 0, "closetimeouts": 0 }
Wed May 17 22:15:05 2017: { "name": "save_tengine", "origin": "core.action", "processed": 0, "failed": 0, "suspended": 0, "suspended.duration": 0, "resumed": 0 }
Wed May 17 22:15:05 2017: { "name": "fwd_tengine_log", "origin": "core.action", "processed": 25, "failed": 0, "suspended": 0, "suspended.duration": 0, "resumed": 0 }
Wed May 17 22:15:05 2017: { "name": "action 10", "origin": "core.action", "processed": 31, "failed": 0, "suspended": 0, "suspended.duration": 0, "resumed": 0 }
Wed May 17 22:15:05 2017: { "name": "action 11", "origin": "core.action", "processed": 6, "failed": 0, "suspended": 0, "suspended.duration": 0, "resumed": 0 }
Wed May 17 22:15:05 2017: { "name": "action 12", "origin": "core.action", "processed": 0, "failed": 0, "suspended": 0, "suspended.duration": 0, "resumed": 0 }
Wed May 17 22:15:05 2017: { "name": "action 13", "origin": "core.action", "processed": 30, "failed": 0, "suspended": 0, "suspended.duration": 0, "resumed": 0 }
Wed May 17 22:15:05 2017: { "name": "action 14", "origin": "core.action", "processed": 0, "failed": 0, "suspended": 0, "suspended.duration": 0, "resumed": 0 }
Wed May 17 22:15:05 2017: { "name": "action 15", "origin": "core.action", "processed": 0, "failed": 0, "suspended": 0, "suspended.duration": 0, "resumed": 0 }
Wed May 17 22:15:05 2017: { "name": "action 16", "origin": "core.action", "processed": 0, "failed": 0, "suspended": 0, "suspended.duration": 0, "resumed": 0 }
Wed May 17 22:15:05 2017: { "name": "resource-usage", "origin": "impstats", "utime": 937857, "stime": 891864, "maxrss": 3968, "minflt": 2664, "majflt": 0, "inblock": 0, "oublock": 28408, "nvcsw": 20007, "nivcsw": 16 }
Wed May 17 22:15:05 2017: { "name": "fwd_squid_log queue[DA]", "origin": "core.queue", "size": 0, "enqueued": 0, "full": 0, "discarded.full": 0, "discarded.nf": 0, "maxqsize": 0 }
Wed May 17 22:15:05 2017: { "name": "fwd_squid_log queue", "origin": "core.queue", "size": 0, "enqueued": 0, "full": 0, "discarded.full": 0, "discarded.nf": 0, "maxqsize": 1 }
Wed May 17 22:15:05 2017: { "name": "fwd_message_log queue[DA]", "origin": "core.queue", "size": 0, "enqueued": 0, "full": 0, "discarded.full": 0, "discarded.nf": 0, "maxqsize": 0 }
Wed May 17 22:15:05 2017: { "name": "fwd_message_log queue", "origin": "core.queue", "size": 0, "enqueued": 1028, "full": 0, "discarded.full": 0, "discarded.nf": 0, "maxqsize": 544 }
Wed May 17 22:15:05 2017: { "name": "fwd_ping_log queue[DA]", "origin": "core.queue", "size": 0, "enqueued": 0, "full": 0, "discarded.full": 0, "discarded.nf": 0, "maxqsize": 0 }
Wed May 17 22:15:05 2017: { "name": "fwd_ping_log queue", "origin": "core.queue", "size": 0, "enqueued": 20, "full": 0, "discarded.full": 0, "discarded.nf": 0, "maxqsize": 20 }
Wed May 17 22:15:05 2017: { "name": "save_ping queue[DA]", "origin": "core.queue", "size": 0, "enqueued": 0, "full": 0, "discarded.full": 0, "discarded.nf": 0, "maxqsize": 0 }
Wed May 17 22:15:05 2017: { "name": "main Q[DA]", "origin": "core.queue", "size": 0, "enqueued": 0, "full": 0, "discarded.full": 0, "discarded.nf": 0, "maxqsize": 0 }
Wed May 17 22:15:05 2017: { "name": "main Q", "origin": "core.queue", "size": 0, "enqueued": 2196, "full": 0, "discarded.full": 0, "discarded.nf": 0, "maxqsize": 1024 }
Wed May 17 22:15:07 2017: { "name": "global", "origin": "dynstats", "values": { } }
Wed May 17 22:15:07 2017: { "name": "imuxsock", "origin": "imuxsock", "submitted": 0, "ratelimit.discarded": 0, "ratelimit.numratelimiters": 0 }
Wed May 17 22:15:07 2017: { "name": "fwd_squid_log", "origin": "core.action", "processed": 0, "failed": 0, "suspended": 0, "suspended.duration": 0, "resumed": 0 }
Wed May 17 22:15:07 2017: { "name": "fwd_message_log", "origin": "core.action", "processed": 847, "failed": 0, "suspended": 0, "suspended.duration": 0, "resumed": 0 }
Wed May 17 22:15:07 2017: { "name": "dynafile cache quesocket", "origin": "omfile", "requests": 0, "level0": 0, "missed": 0, "evicted": 0, "maxused": 0, "closetimeouts": 0 }
Wed May 17 22:15:07 2017: { "name": "save_que", "origin": "core.action", "processed": 0, "failed": 0, "suspended": 0, "suspended.duration": 0, "resumed": 0 }
Wed May 17 22:15:07 2017: { "name": "fwd_que_log", "origin": "core.action", "processed": 122, "failed": 0, "suspended": 0, "suspended.duration": 0, "resumed": 0 }
Wed May 17 22:15:07 2017: { "name": "dynafile cache sanmasocket", "origin": "omfile", "requests": 0, "level0": 0, "missed": 0, "evicted": 0, "maxused": 0, "closetimeouts": 0 }
Wed May 17 22:15:07 2017: { "name": "save_sanma", "origin": "core.action", "processed": 0, "failed": 0, "suspended": 0, "suspended.duration": 0, "resumed": 0 }
Wed May 17 22:15:07 2017: { "name": "fwd_sanma_log", "origin": "core.action", "processed": 0, "failed": 0, "suspended": 0, "suspended.duration": 0, "resumed": 0 }
Wed May 17 22:15:07 2017: { "name": "dynafile cache socket", "origin": "omfile", "requests": 0, "level0": 0, "missed": 0, "evicted": 0, "maxused": 0, "closetimeouts": 0 }
Wed May 17 22:15:07 2017: { "name": "save_tengine", "origin": "core.action", "processed": 0, "failed": 0, "suspended": 0, "suspended.duration": 0, "resumed": 0 }
Wed May 17 22:15:07 2017: { "name": "fwd_tengine_log", "origin": "core.action", "processed": 26, "failed": 0, "suspended": 0, "suspended.duration": 0, "resumed": 0 }
Wed May 17 22:15:07 2017: { "name": "action 8", "origin": "core.action", "processed": 885, "failed": 0, "suspended": 0, "suspended.duration": 0, "resumed": 0 }
Wed May 17 22:15:07 2017: { "name": "action 9", "origin": "core.action", "processed": 0, "failed": 0, "suspended": 0, "suspended.duration": 0, "resumed": 0 }
Wed May 17 22:15:07 2017: { "name": "action 10", "origin": "core.action", "processed": 0, "failed": 0, "suspended": 0, "suspended.duration": 0, "resumed": 0 }
Wed May 17 22:15:07 2017: { "name": "action 11", "origin": "core.action", "processed": 0, "failed": 0, "suspended": 0, "suspended.duration": 0, "resumed": 0 }
Wed May 17 22:15:07 2017: { "name": "action 12", "origin": "core.action", "processed": 0, "failed": 0, "suspended": 0, "suspended.duration": 0, "resumed": 0 }
Wed May 17 22:15:07 2017: { "name": "action 13", "origin": "core.action", "processed": 0, "failed": 0, "suspended": 0, "suspended.duration": 0, "resumed": 0 }
Wed May 17 22:15:07 2017: { "name": "action 14", "origin": "core.action", "processed": 854, "failed": 0, "suspended": 0, "suspended.duration": 0, "resumed": 0 }
Wed May 17 22:15:07 2017: { "name": "resource-usage", "origin": "impstats", "utime": 3648163394, "stime": 2795856965, "maxrss": 178984, "minflt": 5115787, "majflt": 2, "inblock": 5919472, "oublock": 217536128, "nvcsw": 60651761, "nivcsw": 292510 }
Wed May 17 22:15:07 2017: { "name": "fwd_squid_log queue[DA]", "origin": "core.queue", "size": 0, "enqueued": 0, "full": 0, "discarded.full": 0, "discarded.nf": 0, "maxqsize": 450 }
Wed May 17 22:15:07 2017: { "name": "fwd_squid_log queue", "origin": "core.queue", "size": 0, "enqueued": 0, "full": 0, "discarded.full": 0, "discarded.nf": 0, "maxqsize": 1415 }
Wed May 17 22:15:07 2017: { "name": "fwd_message_log queue[DA]", "origin": "core.queue", "size": 0, "enqueued": 0, "full": 0, "discarded.full": 0, "discarded.nf": 0, "maxqsize": 1849781 }
Wed May 17 22:15:07 2017: { "name": "fwd_message_log queue", "origin": "core.queue", "size": 0, "enqueued": 847, "full": 0, "discarded.full": 0, "discarded.nf": 0, "maxqsize": 1500 }
Wed May 17 22:15:07 2017: { "name": "save_que queue[DA]", "origin": "core.queue", "size": 0, "enqueued": 0, "full": 0, "discarded.full": 0, "discarded.nf": 0, "maxqsize": 0 }
Wed May 17 22:15:07 2017: { "name": "save_que queue", "origin": "core.queue", "size": 0, "enqueued": 0, "full": 0, "discarded.full": 0, "discarded.nf": 0, "maxqsize": 0 }
Wed May 17 22:15:07 2017: { "name": "main Q[DA]", "origin": "core.queue", "size": 0, "enqueued": 0, "full": 0, "discarded.full": 0, "discarded.nf": 0, "maxqsize": 0 }
Wed May 17 22:15:07 2017: { "name": "main Q", "origin": "core.queue", "size": 0, "enqueued": 1880, "full": 0, "discarded.full": 0, "discarded.nf": 0, "maxqsize": 181308 }
Wed May 17 22:15:08 2017: { "name": "global", "origin": "dynstats", "values": { } }
Wed May 17 22:15:08 2017: { "name": "imuxsock", "origin": "imuxsock", "submitted": 0, "ratelimit.discarded": 0, "ratelimit.numratelimiters": 0 }
Wed May 17 22:15:08 2017: { "name": "fwd_squid_log", "origin": "core.action", "processed": 0, "failed": 0, "suspended": 0, "suspended.duration": 0, "resumed": 0 }
Wed May 17 22:15:08 2017: { "name": "fwd_message_log", "origin": "core.action", "processed": 995, "failed": 0, "suspended": 0, "suspended.duration": 0, "resumed": 0 }
Wed May 17 22:15:08 2017: { "name": "dynafile cache quesocket", "origin": "omfile", "requests": 0, "level0": 0, "missed": 0, "evicted": 0, "maxused": 0, "closetimeouts": 0 }
Wed May 17 22:15:08 2017: { "name": "save_que", "origin": "core.action", "processed": 0, "failed": 0, "suspended": 0, "suspended.duration": 0, "resumed": 0 }
Wed May 17 22:15:08 2017: { "name": "fwd_que_log", "origin": "core.action", "processed": 131, "failed": 0, "suspended": 0, "suspended.duration": 0, "resumed": 0 }
Wed May 17 22:15:08 2017: { "name": "dynafile cache sanmasocket", "origin": "omfile", "requests": 0, "level0": 0, "missed": 0, "evicted": 0, "maxused": 0, "closetimeouts": 0 }
Wed May 17 22:15:08 2017: { "name": "save_sanma", "origin": "core.action", "processed": 0, "failed": 0, "suspended": 0, "suspended.duration": 0, "resumed": 0 }
Wed May 17 22:15:08 2017: { "name": "fwd_sanma_log", "origin": "core.action", "processed": 0, "failed": 0, "suspended": 0, "suspended.duration": 0, "resumed": 0 }
Wed May 17 22:15:08 2017: { "name": "dynafile cache socket", "origin": "omfile", "requests": 0, "level0": 0, "missed": 0, "evicted": 0, "maxused": 0, "closetimeouts": 0 }
Wed May 17 22:15:08 2017: { "name": "save_tengine", "origin": "core.action", "processed": 0, "failed": 0, "suspended": 0, "suspended.duration": 0, "resumed": 0 }
Wed May 17 22:15:08 2017: { "name": "fwd_tengine_log", "origin": "core.action", "processed": 26, "failed": 0, "suspended": 0, "suspended.duration": 0, "resumed": 0 }
Wed May 17 22:15:08 2017: { "name": "action 8", "origin": "core.action", "processed": 47, "failed": 0, "suspended": 0, "suspended.duration": 0, "resumed": 0 }
Wed May 17 22:15:08 2017: { "name": "action 9", "origin": "core.action", "processed": 0, "failed": 0, "suspended": 0, "suspended.duration": 0, "resumed": 0 }
Wed May 17 22:15:08 2017: { "name": "action 10", "origin": "core.action", "processed": 0, "failed": 0, "suspended": 0, "suspended.duration": 0, "resumed": 0 }
Wed May 17 22:15:08 2017: { "name": "action 11", "origin": "core.action", "processed": 0, "failed": 0, "suspended": 0, "suspended.duration": 0, "resumed": 0 }
Wed May 17 22:15:08 2017: { "name": "action 12", "origin": "core.action", "processed": 0, "failed": 0, "suspended": 0, "suspended.duration": 0, "resumed": 0 }
Wed May 17 22:15:08 2017: { "name": "action 13", "origin": "core.action", "processed": 0, "failed": 0, "suspended": 0, "suspended.duration": 0, "resumed": 0 }
Wed May 17 22:15:08 2017: { "name": "action 14", "origin": "core.action", "processed": 0, "failed": 0, "suspended": 0, "suspended.duration": 0, "resumed": 0 }
protoss5945
Avarage
 
Posts: 14
Joined: Fri May 19, 2017 9:41 am

Re: rsyslog can't send old data

Postby dlang » Tue May 23, 2017 12:00 pm

I don't see a problem, all the DA lines show zero logs in the disk queue. what is not getting sent?
dlang
Frequent Poster
 
Posts: 1001
Joined: Mon Sep 15, 2008 7:44 am

Re: rsyslog can't send old data

Postby protoss5945 » Tue May 23, 2017 12:06 pm

[root@hostname rsyslog]# ll -a
total 140908
drwx------. 2 root root 4096 May 23 19:03 .
drwxr-xr-x. 20 root root 4096 Jun 29 2016 ..
-rw------- 1 root root 3683881 May 17 22:15 action_fwd_message_log.00000056
-rw------- 1 root root 269 May 23 19:03 imfile-state:-usr-local-mallard-pingClient-applog-pingData.log
-rw------- 1 root root 251 May 23 19:03 imfile-state:-usr-local-purge-logs-que.log
-rw------- 1 root root 249 May 23 19:03 imfile-state:-usr-local-sanma-log-access.log
-rw------- 1 root root 256 May 23 18:00 imfile-state:-usr-local-squid-var-logs-error.log
-rw------- 1 root root 254 May 23 19:04 imfile-state:-usr-local-tengine-logs-error.log
-rw------- 1 root root 236 May 23 19:03 imfile-state:-var-log-messages
-rw------- 1 root root 233 May 23 19:03 imfile-state:-var-log-secure
-rw------- 1 root root 240 May 23 18:00 imfile-state:-var-log-useraudit.log
-rw------- 1 root root 7396007 May 23 19:03 stats

the file "action_fwd_message_log.00000056" seems not be sent because if sent rsyslog will delete it


ps: i have two qusetion
1、sometime there is a file named "action_fwd_message_log.qi" what's this file used for?
2、 the log show no data on disk but why the file "action_fwd_message_log.00000056" appeard
Last edited by protoss5945 on Tue May 23, 2017 12:16 pm, edited 1 time in total.
protoss5945
Avarage
 
Posts: 14
Joined: Fri May 19, 2017 9:41 am

Google Ads


Next

Return to Installation

Who is online

Users browsing this forum: No registered users and 0 guests

cron