Apr 172015
 

I noticed this tonight:

{:timestamp=>"2015-04-18T01:01:54.363000+0000", :message=>"Failed parsing date from field", :field=>"syslog_timestamp", :value=>"Apr 18 01:01:49", :exception=>java.lang.IllegalArgumentException: Invalid format: "Apr 18 01:01:49" is malformed at " 18 01:01:49", :level=>:warn}

It seems this is the cause:

{"message":"Apr 18 01:27:18 gelt sshd[41397]: fatal: Read from socket failed: Connection reset by peer [preauth]","@version":"1","@timestamp":"2015-04-18T01:27:25.702Z","type":"syslog","file":"/var/log/messages","host":"gelt.unixathome.org","offset":"1198","tags":["syslog"],"syslog_program":"gelt sshd","syslog_pid":"41397","received_at":"2015-04-18 01:27:25 UTC","received_from":"gelt.unixathome.org","syslog_severity_code":5,"syslog_facility_code":1,"syslog_facility":"user-level","syslog_severity":"notice","@source_host":"%{syslog_hostname}","@message":"fatal: Read from socket failed: Connection reset by peer [preauth]"}

For the record, here are all such instances:

{"message":"Apr 18 01:01:49 www sshd[99654]: fatal: Read from socket failed: Connection reset by peer [preauth]","@version":"1","@timestamp":"2015-04-18T01:01:51.748Z","type":"syslog","file":"/var/log/messages","host":"www.pgcon.org","offset":"95371","tags":["syslog"],"syslog_program":"www sshd","syslog_pid":"99654","received_at":"2015-04-18 01:01:51 UTC","received_from":"www.pgcon.org","syslog_severity_code":5,"syslog_facility_code":1,"syslog_facility":"user-level","syslog_severity":"notice","@source_host":"%{syslog_hostname}","@message":"fatal: Read from socket failed: Connection reset by peer [preauth]"}
{"message":"Apr 18 01:12:18 gelt sshd[40358]: fatal: Read from socket failed: Connection reset by peer [preauth]","@version":"1","@timestamp":"2015-04-18T01:12:20.704Z","type":"syslog","file":"/var/log/messages","host":"gelt.unixathome.org","offset":"967","tags":["syslog"],"syslog_program":"gelt sshd","syslog_pid":"40358","received_at":"2015-04-18 01:12:20 UTC","received_from":"gelt.unixathome.org","syslog_severity_code":5,"syslog_facility_code":1,"syslog_facility":"user-level","syslog_severity":"notice","@source_host":"%{syslog_hostname}","@message":"fatal: Read from socket failed: Connection reset by peer [preauth]"}
{"message":"Apr 18 01:27:18 gelt sshd[41397]: fatal: Read from socket failed: Connection reset by peer [preauth]","@version":"1","@timestamp":"2015-04-18T01:27:25.702Z","type":"syslog","file":"/var/log/messages","host":"gelt.unixathome.org","offset":"1198","tags":["syslog"],"syslog_program":"gelt sshd","syslog_pid":"41397","received_at":"2015-04-18 01:27:25 UTC","received_from":"gelt.unixathome.org","syslog_severity_code":5,"syslog_facility_code":1,"syslog_facility":"user-level","syslog_severity":"notice","@source_host":"%{syslog_hostname}","@message":"fatal: Read from socket failed: Connection reset by peer [preauth]"}
{"message":"Apr 18 01:41:23 news sshd[21254]: fatal: Read from socket failed: Connection reset by peer [preauth]","@version":"1","@timestamp":"2015-04-18T01:41:25.195Z","type":"syslog","file":"/var/log/messages","host":"news.freshports.org","offset":"15241","tags":["syslog"],"syslog_program":"news sshd","syslog_pid":"21254","received_at":"2015-04-18 01:41:25 UTC","received_from":"news.freshports.org","syslog_severity_code":5,"syslog_facility_code":1,"syslog_facility":"user-level","syslog_severity":"notice","@source_host":"%{syslog_hostname}","@message":"fatal: Read from socket failed: Connection reset by peer [preauth]"}
{"message":"Apr 18 01:41:49 www sshd[21373]: fatal: Read from socket failed: Connection reset by peer [preauth]","@version":"1","@timestamp":"2015-04-18T01:41:51.358Z","type":"syslog","file":"/var/log/messages","host":"www.pgcon.org","offset":"95588","tags":["syslog"],"syslog_program":"www sshd","syslog_pid":"21373","received_at":"2015-04-18 01:41:51 UTC","received_from":"www.pgcon.org","syslog_severity_code":5,"syslog_facility_code":1,"syslog_facility":"user-level","syslog_severity":"notice","@source_host":"%{syslog_hostname}","@message":"fatal: Read from socket failed: Connection reset by peer [preauth]"}
{"message":"Apr 18 01:46:49 www sshd[22428]: fatal: Read from socket failed: Connection reset by peer [preauth]","@version":"1","@timestamp":"2015-04-18T01:46:51.358Z","type":"syslog","file":"/var/log/messages","host":"www.pgcon.org","offset":"95688","tags":["syslog"],"syslog_program":"www sshd","syslog_pid":"22428","received_at":"2015-04-18 01:46:51 UTC","received_from":"www.pgcon.org","syslog_severity_code":5,"syslog_facility_code":1,"syslog_facility":"user-level","syslog_severity":"notice","@source_host":"%{syslog_hostname}","@message":"fatal: Read from socket failed: Connection reset by peer [preauth]"}

One day, I’ll come back here and fix that.

Website Pin Facebook Twitter Myspace Friendfeed Technorati del.icio.us Digg Google StumbleUpon Premium Responsive

  3 Responses to “logstash problems”

  1. Hi there! Were you able to fix this issue after all? I’ve got reset connections as well, this always crashes my logstash process… :-(

  2. Hello Dan,

    This may be because you’re using the date filter, and don’t have a match that lines up with the timestamps in your logs.

    Looking at your sample date of Apr 18 01:01:49, I think a match config (to the date filter) that should work is “MMM dd HH:mm:ss”.

    Ref: http://joda-time.sourceforge.net/apidocs/org/joda/time/format/DateTimeFormat.html

    HTH!

    _Khalid