systemd: start service at boot time after network is really up (for WoL purpose)

Question

I have a computer at work which I sometimes wakeup from home in order to access it but when boots and gets another IP address from our DHCP server, how can I access it?

The situation and my “workflow” is as follows:

• From my home PC I connect to the office's VPN
• ssh to a dedicated server in office's LAN (it has a fixed IP address)
• on that server, I call a script that broadcasts a WoL packet with my office PC's MAC address
• my office PC starts up (it really does, for sure!)

Now in theory I'd be able to SSH to my office PC if only I'd knew its IP address. Sometimes it gets the same, sometimes it changes. To circumvent this I had the following idea:

• after booting, my office PC does an SSH to the office server and writes its own IP address into some text file on the server
• I examine that file on the server (which I can connect to because of its fixed IP), find the IP address of my office PC, and can then SSH from my home PC to my office PC

All computers run Linux; Ubuntu 14.04 at home, SLES on the office server, OpenSUSE 13.1 on my office PC. This is all not a problem.

For this all to work I simply need a script on my office PC that runs at boot time when the network is up and running.

My script (publish_ip.sh) is like follows:

# get own IP address:
ip=$(ip addr show | awk '$1=="inet" && $2 !~ /127\.0\.0\.1/ { split($2, a, "/"); print a[1]}');

# SSH to the office server (10.64.5.84) and write own IP address to a file there:
ssh -x -e none 10.64.5.84 "echo $(date) $ip >> office_pc_address.txt"

To run this script at boot time I created a systemd service file, publish-ip.service, for my office PC:

[Unit]
Description=publishes own IP address
Wants=network.target
After=network.target

[Service]
Type=oneshot
ExecStartPre=/usr/bin/sleep 30
ExecStart=/home/perldog/bin/publish_ip.sh
User=perldog

[Install]
WantedBy=multi-user.target

But this is what I always get on my office PC:

linux-tz7m:/usr/lib/systemd/system # systemctl status publish-ip.service
publish-ip.service - publishes own IP address
   Loaded: loaded (/usr/lib/systemd/system/publish-ip.service; enabled)
   Active: failed (Result: exit-code) since Mon 2016-02-29 12:17:34 CET; 4 days ago
  Process: 1688 ExecStart=/home/perldog/bin/publish_ip.sh (code=exited, status=255)
  Process: 1016 ExecStartPre=/usr/bin/sleep 30 (code=exited, status=0/SUCCESS)
 Main PID: 1688 (code=exited, status=255)

Feb 29 12:17:34 linux-tz7m publish_ip.sh[1688]: ssh: connect to host 10.64.5.84 port 22: Network is unreachable

Obviously my service starts and also calls my script but the SSH command in that script fails with Network is unreachable.

I tried everything in my service file so that it runs only after the network is up, but I don't get it. I tried Wants=network.target, After=network.target, WantedBy=multi-user.target, and even inserted an ExecStartPre=/usr/bin/sleep 30. Nothing worked. I always get Network is unreachable when my script is called and tries to SSH to the office server.

Question: What settings are required in my service file so that it runs only after the office server is reachable with SSH?

Note: When I'm at the office and my office PC is up-and-running, both my script and the service work perfectly, i.e. systemctl start publish-ip.service works without any error.

Roden

A related solution, https://mcmap.net/q/587310/-autossh-script-running-as-systemctl-start-service-works-but-systemctl-enable-service-failed. Changing network.target to network-online.target resolved for me. – Bumblebee

Answer 1

I tried all these targets, and they all were reached before DHCP got an IP address. Go figure:

• network-online.target
• remote-fs.target
• nfs-client.target
• dbus.service

What did work was enabling these two:

systemctl enable systemd-networkd.service systemd-networkd-wait-online.service

And then setting

[Unit]
After=systemd-networkd-wait-online.service
Wants=systemd-networkd-wait-online.service

Now it got started after DHCP got an IP address. (A mount point in my case, but could have been your service too)

(On debian9/stretch)

Yeaton

Where would you add the two lines you posted? How would you do to have systemd-networkd-wait-online.service start upon boot? Is it enough with the two added lines? – Sverre

I don't remember exactly, and I've left that job, but i think i just put it in the .service or .mount file. Follow the link to my other post for more information. – Watford

Answer 2

You can delay the starting of your job by adding a ping loop as an ExecStartPre script:

[Service]
   ExecStartPre=/bin/sh -c 'until ping -c1 google.com; do sleep 1; done;'

Found here

Pulque

i tried all the solutions here, despite how hacky it looks, this is what worked. thank you – Rear

In case the network is already up until ... delays execution by 1s. Reverse the logic: /bin/sh -c 'while ! ping -c1 1.1.1.1; do sleep 1; done'. Also, if your ping supports it I'd recommend adding a timeout like w/ -W.5. – Adenectomy

Answer 3

Not a full solution, but I set up an SSH tunnel once the computer boots; it restarts every 5 seconds to prevent getting killed for spamming while the network is still down; you could use something like this paired with your WOL, if you can use the fixed IP system as a bastion host:

[Unit]
Description=Keep open a reverse tunnel to my home server
After=network.target

[Service]
ExecStart=/usr/bin/ssh -NT hometunnel
RestartSec=5
Restart=always

[Install]
WantedBy=multi-user.target

Relevant part of root's .ssh/config:

Host                    hometunnel
HostName                <redacted>
IdentityFile    /root/.ssh/<redacted>
User                    <redacted>
RemoteForward   <redacted> localhost:22
ExitOnForwardFailure yes
ServerAliveInterval 60
ServerAliveCountMax 5

Unbound

Looks like might also try ExecStartPre=/bin/sh -c 'until ping -c1 google.com; do sleep 1; done;'; source: github.com/coreos/bugs/issues/1966#issuecomment-301825679 – Unbound

Answer 4

There used to be a tool called arpwatch which was keeping a log of all networked devices' IP addresses and MAC addresses, so you could simply grep for your computer's MAC and get the IP.

There's also a tool called arping, you can grep through the output for the MAC address.

Another option would be to either keep a random port open on your computer and scan the whole network with nmap for that specific port, or nmap the whole network for hosts that are up and grep for your MAC address...

nmap -sP 192.168.1.0/24

Riggle

Thanks, but neither arpwatch, arping, nor nmap are installed on my office server. :-( I already searched for "how to get IP address from MAC address" but to no result. That's why I had the idea of my approach. I admit it's ridiculously complicated. – Roden

@PerlDog, ARP is to reslove a layer-3 (IP) address to a layer-2 (MAC address) which sounds like the opposite of what you want, based on your comment. There is a place you can look fo MAC address to IP address, and that would be the DHCP server, assuming the host uses DHCP to assign its IP address. – Froehlich

@RonMaupin Thanks, but I tried all those ping -b 10.64.255.255; arp -na | grep $MAC suggestions. None of them worked and I'm pretty sure I don't have "query" access to the DHCP server in our network (I'm non-root). So this publish-ip-script came to my mind but that doesn't work either :-( because it runs before the network is up. – Roden

@PerlDog, IP and ethernet were never designed for LAN discovery. You can get a source (DHCP server) which has the sort of thing you need. Many hosts now have firewalls which can, and often do, block ping and other things which may be used for LAN discovery. A statically assigned host with a software firewall connected to a wired ethernet switch can be almost invisible on the LAN. You could have an application which sends an ARP request for every possible IP address on a LAN, but that could be very large number of addresses and take a long time. – Froehlich

@RonMaupin Sure. And to avoid this, my script kicks in. It's sort of a poor man's DynDNS if you like, i.e. it publishes the IP to a well-known location. I think I should rephrase my question (or post a new one) so it stresses more the systemd/network up issue. I wrote all that stuff because I wanted to avoid having an XY problem. – Roden

Answer 5

the config for networking which controls how "online" gets evaluated

$ cat /etc/default/networking 

# Configuration for networking init script being run during the boot sequence

# Set to 'no' to skip interfaces configuration on boot #CONFIGURE_INTERFACES=yes

# Don't configure these interfaces. Shell wildcards supported/ #EXCLUDE_INTERFACES=

# Set to 'yes' to enable additional verbosity #VERBOSE=no

# Method to wait for the network to become online, # for services that depend on a working network: # - ifup: wait for ifup to have configured an interface. # - route: wait for a route to a given address to appear. # - ping/ping6: wait for a host to respond to ping packets. # - none: don't wait. #WAIT_ONLINE_METHOD=ifup

# Which interface to wait for. # If none given, wait for all auto interfaces, or if there are none, # wait for at least one hotplug interface. #WAIT_ONLINE_IFACE=

# Which address to wait for for route, ping and ping6 methods. # If none is given for route, it waits for a default gateway. #WAIT_ONLINE_ADDRESS=

# Timeout in seconds for waiting for the network to come online. \WAIT_ONLINE_TIMEOUT=10

beside that you may add dependency on unit like ping

ExecStartPre=/bin/bash -c '(while !ping -c1 -w1 1.1.1.1 2>/dev/null; do echo "waiting for ip 1.1.1.1..."; sleep 2; done); echo "up up online!"'

Ambros