Fixing SSH hanging on VPS restart (OpenVZ/Ubuntu)

It’s been a lot of years since I’ve used OpenVZ and had to delve into quirks…

In any case, I recently fired up a new OpenVZ 7 VPS running Ubuntu 18.04 and hit a problem I had seen before: when restarting the VPS from a terminal window via SSH, the terminal would hang for a number of minutes before finally dropping the connection with a “packet_write_wait / broken pipe” message.

Why does this happen?

If you’ve ever lost your internet connection when logged in to an SSH server, you’ve probably seen the same message.

In this situation, the cause is slightly different: the server kills the network connection before stopping the sshd service. So to your local machine, it looks like there’s simply an internet/connection problem. So it waits and waits and finally gives up unless you manually kill the connection.

These little annoyances really hurt the workflow.

Fixing the issue!

Note that commands are done on the VPS/SERVER, and need root privileges, so preface each command with “sudo” where applicable.

The first step I did was the same thing I used in an older Ubuntu version many years ago to solve a shutdown hang problem on a VPS:

apt-get install libpam-systemd dbus

That alone didn’t do it this time around. Since systemd tends to control the order services are started/stopped in these days, I delved into some systemd details. From what I found, it looked like systemd attempts to stop services in the reverse order that it started them.

Now I was getting somewhere! Not long ago I wrote about getting nginx to wait for the network before starting to avoid it failing to start on reboot. Now I just had to get sshd to do the same thing! So here we go:

nano /lib/systemd/system/ssh.service

It looks something like this:

OpenVZ 7 and Ubuntu 18.04 VPS Shutdown or Restart hangs SSH

Add network-online.target to the “After=” line, so that it looks something like this:
Fixed! SSH no longer hangs on reboot or shutdown

Saved, exited, and just a few more terminal commands to run to make sure the new service config file is loaded:

systemctl daemon-reload
systemctl restart ssh.service

The 1st reboot at this point hung, but from the 2nd onward everything worked correctly.

Since sshd now only starts after the network is up, when systemd brings the system down it kills the sshd service before bringing down the network. No more terminal hangs!