I noticed this tonight:
{:timestamp=>"2015-04-18T01:01:54.363000+0000", :message=>"Failed parsing date from field", :field=>"syslog_timestamp", :value=>"Apr 18 01:01:49", :exception=>java.lang.IllegalArgumentException: Invalid format: "Apr 18 01:01:49" is malformed at " 18 01:01:49", :level=>:warn}
It seems this is the cause:
{"message":"Apr 18 01:27:18 gelt sshd[41397]: fatal: Read from socket failed: Connection reset by peer [preauth]","@version":"1","@timestamp":"2015-04-18T01:27:25.702Z","type":"syslog","file":"/var/log/messages","host":"gelt.unixathome.org","offset":"1198","tags":["syslog"],"syslog_program":"gelt sshd","syslog_pid":"41397","received_at":"2015-04-18 01:27:25 UTC","received_from":"gelt.unixathome.org","syslog_severity_code":5,"syslog_facility_code":1,"syslog_facility":"user-level","syslog_severity":"notice","@source_host":"%{syslog_hostname}","@message":"fatal: Read from socket failed: Connection reset by peer [preauth]"}
For the record, here are all such instances:
{"message":"Apr 18 01:01:49 www sshd[99654]: fatal: Read from socket failed: Connection reset by peer [preauth]","@version":"1","@timestamp":"2015-04-18T01:01:51.748Z","type":"syslog","file":"/var/log/messages","host":"www.pgcon.org","offset":"95371","tags":["syslog"],"syslog_program":"www sshd","syslog_pid":"99654","received_at":"2015-04-18 01:01:51 UTC","received_from":"www.pgcon.org","syslog_severity_code":5,"syslog_facility_code":1,"syslog_facility":"user-level","syslog_severity":"notice","@source_host":"%{syslog_hostname}","@message":"fatal: Read from socket failed: Connection reset by peer [preauth]"} {"message":"Apr 18 01:12:18 gelt sshd[40358]: fatal: Read from socket failed: Connection reset by peer [preauth]","@version":"1","@timestamp":"2015-04-18T01:12:20.704Z","type":"syslog","file":"/var/log/messages","host":"gelt.unixathome.org","offset":"967","tags":["syslog"],"syslog_program":"gelt sshd","syslog_pid":"40358","received_at":"2015-04-18 01:12:20 UTC","received_from":"gelt.unixathome.org","syslog_severity_code":5,"syslog_facility_code":1,"syslog_facility":"user-level","syslog_severity":"notice","@source_host":"%{syslog_hostname}","@message":"fatal: Read from socket failed: Connection reset by peer [preauth]"} {"message":"Apr 18 01:27:18 gelt sshd[41397]: fatal: Read from socket failed: Connection reset by peer [preauth]","@version":"1","@timestamp":"2015-04-18T01:27:25.702Z","type":"syslog","file":"/var/log/messages","host":"gelt.unixathome.org","offset":"1198","tags":["syslog"],"syslog_program":"gelt sshd","syslog_pid":"41397","received_at":"2015-04-18 01:27:25 UTC","received_from":"gelt.unixathome.org","syslog_severity_code":5,"syslog_facility_code":1,"syslog_facility":"user-level","syslog_severity":"notice","@source_host":"%{syslog_hostname}","@message":"fatal: Read from socket failed: Connection reset by peer [preauth]"} {"message":"Apr 18 01:41:23 news sshd[21254]: fatal: Read from socket failed: Connection reset by peer [preauth]","@version":"1","@timestamp":"2015-04-18T01:41:25.195Z","type":"syslog","file":"/var/log/messages","host":"news.freshports.org","offset":"15241","tags":["syslog"],"syslog_program":"news sshd","syslog_pid":"21254","received_at":"2015-04-18 01:41:25 UTC","received_from":"news.freshports.org","syslog_severity_code":5,"syslog_facility_code":1,"syslog_facility":"user-level","syslog_severity":"notice","@source_host":"%{syslog_hostname}","@message":"fatal: Read from socket failed: Connection reset by peer [preauth]"} {"message":"Apr 18 01:41:49 www sshd[21373]: fatal: Read from socket failed: Connection reset by peer [preauth]","@version":"1","@timestamp":"2015-04-18T01:41:51.358Z","type":"syslog","file":"/var/log/messages","host":"www.pgcon.org","offset":"95588","tags":["syslog"],"syslog_program":"www sshd","syslog_pid":"21373","received_at":"2015-04-18 01:41:51 UTC","received_from":"www.pgcon.org","syslog_severity_code":5,"syslog_facility_code":1,"syslog_facility":"user-level","syslog_severity":"notice","@source_host":"%{syslog_hostname}","@message":"fatal: Read from socket failed: Connection reset by peer [preauth]"} {"message":"Apr 18 01:46:49 www sshd[22428]: fatal: Read from socket failed: Connection reset by peer [preauth]","@version":"1","@timestamp":"2015-04-18T01:46:51.358Z","type":"syslog","file":"/var/log/messages","host":"www.pgcon.org","offset":"95688","tags":["syslog"],"syslog_program":"www sshd","syslog_pid":"22428","received_at":"2015-04-18 01:46:51 UTC","received_from":"www.pgcon.org","syslog_severity_code":5,"syslog_facility_code":1,"syslog_facility":"user-level","syslog_severity":"notice","@source_host":"%{syslog_hostname}","@message":"fatal: Read from socket failed: Connection reset by peer [preauth]"}
One day, I’ll come back here and fix that.
Hi there! Were you able to fix this issue after all? I’ve got reset connections as well, this always crashes my logstash process… :-(
This is not about reset connections. It’s about ‘Failed parsing date from field’.
Sorry, but I don’t think we’re dealing with the same problem.
Hello Dan,
This may be because you’re using the date filter, and don’t have a match that lines up with the timestamps in your logs.
Looking at your sample date of Apr 18 01:01:49, I think a match config (to the date filter) that should work is “MMM dd HH:mm:ss”.
Ref: http://joda-time.sourceforge.net/apidocs/org/joda/time/format/DateTimeFormat.html
HTH!
_Khalid