Archive | December 2011

fstab, boot, mount and STP: a big challenge to solve

Last week I have to fight with this problem: my host cannot mount by fstab at boot time but it mounts without problem after login by mount -a.
Syslog reports this error:

mount error 113 = No route to host
Refer to the mount.cifs(8) manual page ( mount.cifs)

Online you can find too many posts about that error:

  • fstab not mounting my cifs, I have to manually call mount -a
  • unable to mount cifs (windows) partition through fstab
  • cifs not mounting on boot

In fact, it just reports that your host cannot contact your share-server ip.
And so we are going to troubleshoot…

1) Networking problem on host

Of course we focus on networking configuration of host: what’s the difference between manually call mount -a and fstab? When you manually mount, your network interface it’s already up (and you can easly test it with a ping to your server). But what about boot time?
Just look a /var/log/syslog and see what happens before your mount error alerts.
If your network configuration (/etc/network/interfaces or NetworkManager with “all users” automatic connection) it’s ok you can see something like “eth0 up”, “link up”, “eth0 ready” before mount errors! It seems ok!
To be sure, your can also use a fixed ip, but the problem remains!
My host use a fixed ip and syslog reports that my interface (eth0) it’s up before mount try to contact the remote server! WTF!?!? It use a fixed ip. It has eth0 up. Network setup it’s ok. This can make you(&me) crazy!

2) Networking problem on cable (bottom/physical layer)

If your network configuration is ok and up at mount time where is that fu**ing problem!? You can have a problem like this if you use a static ip address, a permanent up interface (without hotplug) but your cable is not connected. But your cable it’s just connected!!!
You can think that a mystical problem involves your packets but it’s strange that it happens only at boot time, after boot all is working fine!
As usual by IT, nothing is mystical: put your network tester in the closet.

3) No problem on host, no problem on cable, what’s next? Switch!

Modern switches uses the STP (Spanning Tree Protocol) on all ports by default to prevent bridge loops and the broadcast radiation.
When a device is first attached to a switch (STP enabled) port, it will not immediately start to forward data. It will instead go through a number of states while it determines the topology of the network. When a host is attached such as a computer, printer or server the port will always go into the forwarding state, albeit after a delay of about 30 seconds while it goes through the listening and learning states. Bingo!
Our interface it’s up and its configuration is ok but our switch don’t forward our host’s packets for about 30 seconds and so we cannot mount by fstab at boot!

4) Solution

A switch with STP is probably managed and your can configure STP properties.
My switch is able to disable STP globally or on a specific port.
Because STP is completely useless for a generic host (and only wasted my time) I have turn STP off on port of host. You can do so safely.
If you cannot turn off STP by port your can disable it globally but be sure that you don’t need it (or booommm).
If you cannot turn off STP on your switch (globally or by port) you can lower the “hello time” STP param to 1 (default is 2) but this can only reduce the listening/learning time (your host may always try to mount before).
If you dont want to turn off STP (or you can’t) you can remove the “auto” option from fstab and put a mount command like sleep 30 && mount /mnt/myshare & in your /etc/rc.local (sleep ensures that your mount happens after 30 seconds when your STP switch port starts to forward).

Happy hacking!!!

Note: 30 seconds it’s just an indication, STP can take usually 30 to 50 seconds to respond to a topology change. If your switch uses  RSTP (Rapid Spanning Tree Protocol) it’s typically able to respond to changes within 3 × Hello times (default: 3 times 2 seconds).