from Centos system log to GELF

General discussions here

Moderator: rgerhards

Google Ads


from Centos system log to GELF

Postby rolele » Mon Oct 24, 2016 4:39 am

I have installed rsyslog as a service on my centos machine 192.168.77.20 and I have a logstash aggregator on another machine 192.168.77.21

my logstash input is listening on udp 12201 with GELF format (I also added a simple udp input on 12202 for debugging purposes)
Code: Select all
input {
  gelf {
    type => docker
    port => 12201
  }
  udp {
    port => 12202
  }
}

output {
  stdout {
  }
  elasticsearch {
    hosts => "elasticsearch:9200"
  }
}


on the rsyslog machine I have this config with the GELF template (and the plain log for debugging purposes)
Code: Select all
template(name="gelf" type="list") {
        constant(value="{\"version\":\"1.1\",")
        constant(value="\"host\":\"")
        property(name="hostname")
        constant(value="\",\"short_message\":\"")
        property(name="msg" format="json")
        constant(value="\",\"timestamp\":\"")
        property(name="timegenerated" dateformat="unixtimestamp")
        constant(value="\",\"level\":\"")
        property(name="syslogseverity")
        constant(value="\"}")
}
*.* @192.168.77.21:12201;gelf
*.* @192.168.77.21:12202



this is the log of my logstash instance (there is 3 logs here)
Code: Select all
{:timestamp=>"2016-10-24T03:16:03.525000+0000", :message=>"Gelfd failed to parse a message skipping", :exception=>#<Gelfd::UnknownHeaderError: Could not find parser for header: [123, 34]>, :backtrace=>["/opt/logstash/vendor/bundle/jruby/1.9/gems/gelfd-0.2.0/lib/gelfd/parser.rb:14:in `parse'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-gelf-2.0.7/
lib/logstash/inputs/gelf.rb:104:in `udp_listener'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-gelf-2.0.7/lib/logstash/inputs/gelf.rb:77:in `run'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.4.0-java/lib/logstash/pipeline.rb:342:in `inputworker'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.4.0-java/lib/logstash/p
ipeline.rb:336:in `start_input'"], :level=>:warn}
{:timestamp=>"2016-10-24T03:16:03.536000+0000", :message=>"Gelfd failed to parse a message skipping", :exception=>#<Gelfd::UnknownHeaderError: Could not find parser for header: [123, 34]>, :backtrace=>["/opt/logstash/vendor/bundle/jruby/1.9/gems/gelfd-0.2.0/lib/gelfd/parser.rb:14:in `parse'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-gelf-2.0.7/lib/logstash/inputs/gelf.rb:104:in `udp_listener'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-gelf-2.0.7/lib/logstash/inputs/gelf.rb:77:in `run'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.4.0-java/lib/logstash/pipeline.rb:342:in `inputworker'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.4.0-java/lib/logstash/p
ipeline.rb:336:in `start_input'"], :level=>:warn}
{:timestamp=>"2016-10-24T03:16:03.542000+0000", :message=>"Gelfd failed to parse a message skipping", :exception=>#<Gelfd::UnknownHeaderError: Could not find parser for header: [123, 34]>, :backtrace=>["/opt/logstash/vendor/bundle/jruby/1.9/gems/gelfd-0.2.0/lib/gelfd/parser.rb:14:in `parse'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-gelf-2.0.7/lib/logstash/inputs/gelf.rb:104:in `udp_listener'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-gelf-2.0.7/lib/logstash/inputs/gelf.rb:77:in `run'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.4.0-java/lib/logstash/pipeline.rb:342:in `inputworker'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.4.0-java/lib/logstash/p
ipeline.rb:336:in `start_input'"], :level=>:warn}
2016-10-24T03:16:03.525Z 10.255.0.4 <85>Oct 24 04:16:03 manager1 sudo: vagrant : TTY=pts/1 ; PWD=/home/vagrant ; USER=root ; COMMAND=/bin/su
2016-10-24T03:16:03.537Z 10.255.0.4 <37>Oct 24 04:16:03 manager1 su: (to root) vagrant on pts/1
2016-10-24T03:16:03.544Z 10.255.0.4 <86>Oct 24 04:16:03 manager1 su: pam_unix(su:session): session opened for user root by vagrant(uid=0)


you can see that it failed on the GELF format and you see the actual log that was sent without rsyslog templating.

What am I doing wrong?
How can I send gelf formatted log to logstash?
rolele
New
 
Posts: 1
Joined: Mon Oct 24, 2016 4:25 am

Urgent Question?

  • Pulling out your Hair?
  • Wasting Time and Money?
  • Deadline Approaching?

Google Ads


Return to General

Who is online

Users browsing this forum: No registered users and 2 guests

cron