How to monitor multiples files using Read Event from file ?

Support, Questions and Discussions on EventReporter

Moderator: alorbach

Google Ads


How to monitor multiples files using Read Event from file ?

Postby xfrsquad42 » Wed Mar 12, 2014 2:39 pm

Hello all,

EventReporter version : 13.1.367
Platform where EventReporter is intalled : Windows Server 2008 R2 X64
Netapp Filer Version : Data Ontap 7.3.4

We want to collect Netapp filers audit logs using your product (we are in the testing stage)

Netapp audit logs are stored in a cifs share with the following form : access_audit.YYYYMMDDHHMMSS.evt

So, in EventReporter, I chose Read Event from log file and in File & Path Name I entered this : \\filer01\audit\logs\%Y%m%d%h%M%S*.evt
But it doesn't work...nothing is sent via syslog.

Here is the debug log :

CInfoSourceNTEventReport|ShowEvtLog: Start checking the 'Security' Channel.
RegistryRead:DWORD:SOFTWARE\Adiscon\EventReporter\Services\IS2\nConfigVersion:'2' (state 1)
RegistryRead:STRING:SOFTWARE\Adiscon\EventReporter\Services\IS2\szRuleSetName:'Default RuleSet' (state 1)
RegistryRead:STRING:SOFTWARE\Adiscon\EventReporter\Services\IS2\szSyslogTagValue:'EvntSLog' (state 1)
RegistryRead:DWORD:SOFTWARE\Adiscon\EventReporterData\Services\{E1584C94-B7BF-4021-8272-4BEDB5935CC3}\Security\nLastSyslogMsgNbr:'0' (state 0)
RegistryRead:QWORD:SOFTWARE\Adiscon\EventReporterData\Services\{E1584C94-B7BF-4021-8272-4BEDB5935CC3}\Security\nLastSyslogMsgNbr: DEFAULT ('0') used.
RegistryRead:DWORD:SOFTWARE\Adiscon\EventReporterData\Services\{E1584C94-B7BF-4021-8272-4BEDB5935CC3}\Security\dwLastRecordCrc:'1354767717' (state 1)
RegistryRead:DWORD:SOFTWARE\Adiscon\EventReporterData\Services\{E1584C94-B7BF-4021-8272-4BEDB5935CC3}\Security\nLastRecord:'1' (state 1)
CInfoSourceNTEventReport|ShowEvtLog: Could not Open the EventLog Profile 'Security' - GetLastError = 123
Error Event 1003 logged with data 'Security'.
CInfoSourceNTEventReport|RUN: finished the run.


Nevertheless, if I choose a specific evt file in File & Path Name , for exemple \\filer01\audit\logs\20140304223122.evt it works

Any suggestions regarding this issue ?

Thx for your help :-)
xfrsquad42
Avarage
 
Posts: 14
Joined: Wed Mar 12, 2014 2:28 pm

Re: How to monitor multiples files using Read Event from fil

Postby friedl » Wed Mar 12, 2014 3:40 pm

Please try *.evt

Wildcards should work fine in filenames, but properties do not.

Florian
friedl
Adiscon Support
 
Posts: 67
Joined: Wed Sep 13, 2006 2:31 pm

Re: How to monitor multiples files using Read Event from fil

Postby xfrsquad42 » Wed Mar 12, 2014 5:21 pm

Hi friedl, thx for your reply

I tried \\filer01\audit\logs\*.evt but it failed again

For information, here is Netapp timestamp specifications :

The timestamp file name is constructed using the following variables:

base_name_of_event_file.YYYYMMDDHHMMSS.evt

Variable Description
YYYY The 4-digit year
MM The 2-digit month
DD The 2-digit day
HH The 2-digit hour
MM The 2-digit minute
SS The 2-digit second
xfrsquad42
Avarage
 
Posts: 14
Joined: Wed Mar 12, 2014 2:28 pm

Re: How to monitor multiples files using Read Event from fil

Postby alorbach » Wed Mar 12, 2014 6:20 pm

Have you tried the "Enable date replacement characters" option? It is below "Type of Eventlog" and was especially added for NetApp.
Take a look to the manual for more details: http://www.eventreporter.com/en/manual/ ... nitor1.htm

best regards,
Andre Lorbach
alorbach
Site Admin
 
Posts: 1627
Joined: Thu Feb 13, 2003 11:55 am

Re: How to monitor multiples files using Read Event from fil

Postby xfrsquad42 » Thu Mar 13, 2014 9:43 am

I tried the option "Enable date replacement characters" but it failed again

I tried to reduce the file name of logs, to just have a name format like this : audit.YYYYMMDD.evt (for example audit.20140131.evt) but same issue

Does somebody have already tested to read event from different file ? A feedback would be welcome ;-)
xfrsquad42
Avarage
 
Posts: 14
Joined: Wed Mar 12, 2014 2:28 pm

Re: How to monitor multiples files using Read Event from fil

Postby friedl » Thu Mar 13, 2014 10:08 am

Does the user account which the EventReporter service runs with have suffient rights to access the NetApp Eventlog files, preferably a admin account?
friedl
Adiscon Support
 
Posts: 67
Joined: Wed Sep 13, 2006 2:31 pm

Re: How to monitor multiples files using Read Event from fil

Postby alorbach » Thu Mar 13, 2014 10:10 am

Errorcode 123 means "ERROR_INVALID_NAME" btw, so it must be an issue with the configured filename itself.
Can you please post your full configured file and path name?

best regards,
Andre Lorbach
alorbach
Site Admin
 
Posts: 1627
Joined: Thu Feb 13, 2003 11:55 am

Re: How to monitor multiples files using Read Event from fil

Postby xfrsquad42 » Thu Mar 13, 2014 10:38 am

@friedl : I don't connect directly to Netapp filer. I just use a CIFS share to access log files.
Rights on log files are : Everyone with permissions to Read,Write, Execute etc....And when I select a specific log file, no problem with that so I don't think it's a permission issue

@alorbach :

File is at this location : \\filer01\audit$\log\access_audit.20140313095531.evt

The Pathname I put in Event Reporter : \\filer01\audit$\log\access_audit.%Y%m%d%h%M%S.evt

(I tried also \\filer01\audit$\log\access_audit.%Y%m%d%h%M%S*.evt (* added at the end) like we see here http://www.rsyslog.com/how-to-setup-rsyslog-windows-agent-to-monitor-netapp-devices-using-backup-evt-files/ but same issue


EDIT : Something realy strange :

I tried with a log format name like this : access_audit.YYYYMM.evt

I put 2 files with this log format name : access_audit_201403.evt and access_audit_201404.evt

The first log file ( access_audit_201403.evt) is well processed :

Code: Select all
CInfoSourceNTEventReport|ShowEvtLog: Dynamic Filename generated '\\filer01\audit$\log\access_audit.201403.evt'
CInfoSourceNTEventReport|ShowEvtLog: Failed to open Eventlog Backupfile, trying FALLBACK - manually read and parse the file
CInfoSourceNTEventReport|DoReadEventLogFromFile: Reached END of the Eventlogfile at RecordID '31'
CInfoSourceNTEventReport|ShowEvtLog: Check for 'Security' Channel finished!
CInfoSourceNTEventReport|RUN: finished the run.


But not the second one :

Code: Select all
Event Log Monitor is starting a scheduled run.
CInfoSourceNTEventReport|ShowEvtLog: Start checking the 'Security' Channel.
RegistryRead:DWORD:SOFTWARE\Adiscon\EventReporter\Services\IS4\nConfigVersion:'1' (state 1)
RegistryRead:STRING:SOFTWARE\Adiscon\EventReporter\Services\IS4\szRuleSetName:'Default RuleSet' (state 1)
RegistryRead:STRING:SOFTWARE\Adiscon\EventReporter\Services\IS4\szSyslogTagValue:'EvntSLog' (state 1)
RegistryRead:DWORD:SOFTWARE\Adiscon\EventReporterData\Services\{FEAA8470-0A1E-4E06-9A99-E11D992DDC76}\Security\nLastSyslogMsgNbr:'0' (state 0)
RegistryRead:QWORD:SOFTWARE\Adiscon\EventReporterData\Services\{FEAA8470-0A1E-4E06-9A99-E11D992DDC76}\Security\nLastSyslogMsgNbr: DEFAULT ('0') used.
RegistryRead:DWORD:SOFTWARE\Adiscon\EventReporterData\Services\{FEAA8470-0A1E-4E06-9A99-E11D992DDC76}\Security\dwLastRecordCrc:'0' (state 0)
RegistryRead:QWORD:SOFTWARE\Adiscon\EventReporterData\Services\{FEAA8470-0A1E-4E06-9A99-E11D992DDC76}\Security\dwLastRecordCrc: DEFAULT ('0') used.
RegistryRead:DWORD:SOFTWARE\Adiscon\EventReporterData\Services\{FEAA8470-0A1E-4E06-9A99-E11D992DDC76}\Security\nLastRecord:'0' (state 0)
RegistryRead:QWORD:SOFTWARE\Adiscon\EventReporterData\Services\{FEAA8470-0A1E-4E06-9A99-E11D992DDC76}\Security\nLastRecord: DEFAULT ('0') used.
CInfoSourceNTEventReport|ShowEvtLog: Could not Open the EventLog Profile 'Security' - GetLastError = 123
Error Event 1003 logged with data 'Security'.
CInfoSourceNTEventReport|RUN: finished the run.
xfrsquad42
Avarage
 
Posts: 14
Joined: Wed Mar 12, 2014 2:28 pm

Re: How to monitor multiples files using Read Event from fil

Postby alorbach » Thu Mar 13, 2014 11:02 am

Can you enable full debug output to see, what filename EventReporter generates before it tries to open the Eventlog file?
Also use this filename instead: \\filer01\audit$\log\access_audit.%Y%m%d%*.evt

We had this issue earlier, putting hours, minutes and seconds into the filename wasn't a good Idea by netapp.

best regards,
Andre Lorbach
alorbach
Site Admin
 
Posts: 1627
Joined: Thu Feb 13, 2003 11:55 am

Re: How to monitor multiples files using Read Event from fil

Postby xfrsquad42 » Thu Mar 13, 2014 1:54 pm

Here is output debug when It works :

Code: Select all
CInfoSourceNTEventReport|ShowEvtLog: Dynamic Filename generated '\\filer01\audit$\log\access_audit.201403*.evtx'
CInfoSourceNTEventReport|ShowEvtLog: Found dynamic file by wildcard: '\\filer01\audit$\log\access_audit.201403.evtx'
CInfoSourceNTEventReport|ShowEvtLog: Security: dwRead: 428, RLength: 428, RecNum: 1 (nSleepTime 60000, bAbortFlag 0)
CInfoSourceNTEventReport|ShowEvtLog: New nLastRecord: 1, bAbortFlag: 0
CInfoSourceNTEventReport|DoReadEventLog: ReadEventLog reached END of the Eventlog at RecordID '1'
CInfoSourceNTEventReport|ShowEvtLog: Check for 'Security' Channel finished!


Another log line interesting :

Code: Select all
RegistryRead:STRING:SOFTWARE\Adiscon\EventReporter\Services\IS2\Security\szLogFileName:'\\filer01\audit$\log\access_audit.%Y%m*.evt' (state 1)


What I don't understand is that there is not any error message for the second log file access_audit.201404.evtx'

I tried \\filer01\audit$\log\access_audit.%Y%m%d%*.evt but it failed
I tried \\filer01\audit$\log\access_audit.%Y%m%d*.evt but it failed

And now, it didn't work no more with \\filer01\audit$\log\access_audit.%Y%m.evt (it worked a half before) :-/

Can you reproduce the same issue ?
xfrsquad42
Avarage
 
Posts: 14
Joined: Wed Mar 12, 2014 2:28 pm

Re: How to monitor multiples files using Read Event from fil

Postby alorbach » Thu Mar 13, 2014 3:04 pm

We do not have a netapp device for testing to reproduce this problem.
However I am a little bit confused right now, your debugfile shows a ".evtx" file and not a ".evt".
Does your netapp device generate Windows 2008 evtx files?

I would say just use this as file and path name: \\filer01\audit$\log\access_audit.%Y%m%d%*

best regards,
Andre Lorbach
alorbach
Site Admin
 
Posts: 1627
Joined: Thu Feb 13, 2003 11:55 am

Re: How to monitor multiples files using Read Event from fil

Postby xfrsquad42 » Thu Mar 13, 2014 3:15 pm

By reproducing the issue, I think to test the "Read Event log file" function locally, no need Netapp filer at all (on your computer for example, create 2 evt files then check if you can reproduce the same issue when using date replacement function : %Y%m.evt)

Don't care about evtx, Netapp generate evt but for testing purposes, I put .evtx in \\filer01\audit$\log (because LogParser can't work with evt, they have to be converted in evtx but that's another story ;-)
xfrsquad42
Avarage
 
Posts: 14
Joined: Wed Mar 12, 2014 2:28 pm

Re: How to monitor multiples files using Read Event from fil

Postby alorbach » Thu Mar 13, 2014 3:31 pm

xfrsquad42 wrote:I tried \\filer01\audit$\log\access_audit.%Y%m%d%*.evt but it failed


Can't work, one % to much.

xfrsquad42 wrote:I tried \\filer01\audit$\log\access_audit.%Y%m%d*.evt but it failed


Should work if there is a .evt file with the date of today. However we had the case that the evt files generated by netapp are one day off.
Therefor, you will find an offset option below the Enable date replacement characters. To generate filenames matching filenames from yesterday, use -84600 as offset.

best regards,
Andre Lorbach
alorbach
Site Admin
 
Posts: 1627
Joined: Thu Feb 13, 2003 11:55 am

Re: How to monitor multiples files using Read Event from fil

Postby xfrsquad42 » Fri Mar 14, 2014 11:41 am

Hello all,

I'm back with some tests :

I tested many patterns using these filename :

Code: Select all
    access_audit.201403.evtx
    access_audit.20140314.evtx
    access_audit.2014031408.evtx
    access_audit.201403140827.evtx
    access_audit.20140314082751.evtx
    access_audit.201403140828.evtx
    access_audit.20140314082852.evtx
    access_audit.2014031409.evtx
    access_audit.20140315.evtx
    access_audit.201404.evtx


Here is the results :

Imageimagik

Looking at the results, it means that :

- EventReporter can't handle minutes and secondes in filename path
- EventReporter need Date Replacement option to process log file

But why Event Reporter just process only one log file on the two ?


I also found something weird :

Using the pattern %Y%m%d%h%M%S and the option Date Replacement, if you look at the debug log :

(I created a neew debug file for each test, and I just put the more important)

Code: Select all
Service STARTUP [V13.1(Final Release)]- the time is Fri Mar 14 10:16:00 2014
RegistryRead:STRING:SOFTWARE\Adiscon\EventReporter\Services\IS2\Security\szLogFileName:'C:\netapp\access_audit.%Y%m%d%h%M%S.evtx' (state 1)
CInfoSourceNTEventReport|ShowEvtLog: Dynamic Filename generated 'C:\netapp\access_audit.20140314091600.evtx'
CInfoSourceNTEventReport|ShowEvtLog: Could not Open the EventLog Profile 'Security' - GetLastError = 2


Looks closely to Service startup date : this is Fri Mar 14 10:16:00 2014
Now looks closely to "Dynamic Filename generated" : 'C:\netapp\access_audit.20140314091600.evtx' . This means Fri Mar 14 09:16:00 2014. So exactly 1 hour different with the Service statup date which is Fri Mar 14 10:16:00 2014

Why EventReporter take for reference the date of EventReporter Service Startup ? (This has been observed many times)


ANd does Date Replacement option only works with actual date/hours ? For exemple, it would not work with a log file of 2012 ?


What is your opinion about that ? Thx for your support
xfrsquad42
Avarage
 
Posts: 14
Joined: Wed Mar 12, 2014 2:28 pm

Re: How to monitor multiples files using Read Event from fil

Postby alorbach » Fri Mar 14, 2014 1:02 pm

Perhaps there is a misunderstanding how EventReporter works here.
The date replacement options were added to be able to process the latest found and matching logfile from the working directory.

I think you assume that EventReporter should process all files matching your filename definitions. This is NOT the case. EventReporter is designed to work on a single logfile only.

Isn't it possible to limit your netapp device to write one eventlog file per day? This would make processing much more easier.

best regards,
Andre Lorbach
alorbach
Site Admin
 
Posts: 1627
Joined: Thu Feb 13, 2003 11:55 am

Google Ads


Next

Return to EventReporter

Who is online

Users browsing this forum: No registered users and 1 guest

cron