Why is my network offline? An investigation

Happy New Year.

This morning I received an email at 4:50 AM local time telling me that my camera system had been offline for 10 minutes. Based on that, I tried to access my internal monitoring system from outside. Nothing.

Later emails indicated other external systems had issues connecting to the internal network.

As I type this, I just arrived at home, I’m sitting on the couch and I’ve not touched the network at all. I’m using a mobile hotspot, not related to my home network.

I thought you might find this interesting… but yeah, it’s not as exciting as my mind made it out to be.

Hypothesis

There are a few ideas I have as to why the network is offline:

  • My home IP address has changed and the dynamic DNS has not updated.
  • I recently connected my ISP equipment to my switch, instead of directly into the firewall. Perhaps my ISP has noticed this and is refusing to renew the DHCP lease based on the changed MAC address.
  • The ISP connection is dead.

Can I get out?

I asked Alexa (that link contains my Amazon referrer code) what time it was. She said “Sorry, I’m having trouble understanding right now”. OK, Alexa cannot connect to the internet. This probably means it’s the firewall, my pfSense box.

What about my laptop?

My laptop had been left powered on. The most recent email found there was from 4:14 AM. Note that it did not include the camera notification from 4:50 AM mentioned above.

Conclusion: The network cannot get out, nor can anything get in.

Can my laptop ping the gateway?

Is the network functional? Can my laptop ping the gateway?

In this context, my gateway is also my firewall. The two terms can both be used to describe my pfSense box, but not all gateways are firewalls and not all firewalls are gateways.

Yes it can.

ping works
ping works

The gateway

Next, I got only my pfSense webui.

As soon as I logged in, I could see that the WAN (i.e. my ISP connection) had an IP address of 0.0.0.0

WAN 0.0.0.0
WAN 0.0.0.0

That makes me think about the MAC address theory I mentioned earlier.

Renewing the lease

I could not find a place to renew the WAN DHCP lease. I could find nowhere to restart dhclient.

I was directed to ‘status > dhcp leases > press the trash button in the actions column for that specific IP’ but that page listed no leases. I also think that page is for leases this pfSense instance has handed out, not those which it has been issued.

To restart dhclient, I went to Interfaces > WAN, changed IPv4 Configuration Type to None, and clicked on Save. This stopped dhclient.

I changed the value back to DHCP, clicked on Save, and dhclient started up again.

dclient eventually reported FAIL.

The router

In this context, when I say router, I mean the equipment supplied by my ISP.

I power cycled it and watched dhclient. Still no lease.

I still suspect the issue is: My ISP knows my MAC address and does not want to issue another DHCP lease to anything else. I think I will need to:

  1. plug the router directly into my pfSense box
  2. let it get a lease
  3. release that lease manually
  4. plug the router back into the switch
  5. get a new lease

I tried the above; I was still unable to get a lease.

I power cycled the router for a second time.

Nothing.

Let’s go outside

I went out and checked the ONT. Mine had only the power light. My neighbor’s had PWR, BTRY DATA NTWK MGMT, and MoCA.

ONT power light
ONT power light

Called Verizon

I contacted Verizon Support via a DM on Twitter. I got a response within minutes.

I talked to support via telephone. They are sending a tech out on Wednesday.

Yes, today is a statutory holiday. Yes, it’s going to be 48 hours to get them here.

I am glad it’s not something I did.

I am not happy it will take over 48 hours to get someone here.

Website Pin Facebook Twitter Myspace Friendfeed Technorati del.icio.us Digg Google StumbleUpon Premium Responsive

Leave a Comment

Scroll to Top