fstab, boot, mount and STP: a big challenge to solve

Last week I have to fight with this problem: my host cannot mount by fstab at boot time but it mounts without problem after login by mount -a.
Syslog reports this error:

mount error 113 = No route to host
Refer to the mount.cifs(8) manual page (e.g.man mount.cifs)

Online you can find too many posts about that error:

  • fstab not mounting my cifs, I have to manually call mount -a
  • unable to mount cifs (windows) partition through fstab
  • cifs not mounting on boot

In fact, it just reports that your host cannot contact your share-server ip.
And so we are going to troubleshoot…

1) Networking problem on host

Of course we focus on networking configuration of host: what’s the difference between manually call mount -a and fstab? When you manually mount, your network interface it’s already up (and you can easly test it with a ping to your server). But what about boot time?
Just look a /var/log/syslog and see what happens before your mount error alerts.
If your network configuration (/etc/network/interfaces or NetworkManager with “all users” automatic connection) it’s ok you can see something like “eth0 up”, “link up”, “eth0 ready” before mount errors! It seems ok!
To be sure, your can also use a fixed ip, but the problem remains!
My host use a fixed ip and syslog reports that my interface (eth0) it’s up before mount try to contact the remote server! WTF!?!? It use a fixed ip. It has eth0 up. Network setup it’s ok. This can make you(&me) crazy!

2) Networking problem on cable (bottom/physical layer)

If your network configuration is ok and up at mount time where is that fu**ing problem!? You can have a problem like this if you use a static ip address, a permanent up interface (without hotplug) but your cable is not connected. But your cable it’s just connected!!!
You can think that a mystical problem involves your packets but it’s strange that it happens only at boot time, after boot all is working fine!
As usual by IT, nothing is mystical: put your network tester in the closet.

3) No problem on host, no problem on cable, what’s next? Switch!

Modern switches uses the STP (Spanning Tree Protocol) on all ports by default to prevent bridge loops and the broadcast radiation.
When a device is first attached to a switch (STP enabled) port, it will not immediately start to forward data. It will instead go through a number of states while it determines the topology of the network. When a host is attached such as a computer, printer or server the port will always go into the forwarding state, albeit after a delay of about 30 seconds while it goes through the listening and learning states. Bingo!
Our interface it’s up and its configuration is ok but our switch don’t forward our host’s packets for about 30 seconds and so we cannot mount by fstab at boot!

4) Solution

A switch with STP is probably managed and your can configure STP properties.
My switch is able to disable STP globally or on a specific port.
Because STP is completely useless for a generic host (and only wasted my time) I have turn STP off on port of host. You can do so safely.
If you cannot turn off STP by port your can disable it globally but be sure that you don’t need it (or booommm).
If you cannot turn off STP on your switch (globally or by port) you can lower the “hello time” STP param to 1 (default is 2) but this can only reduce the listening/learning time (your host may always try to mount before).
If you dont want to turn off STP (or you can’t) you can remove the “auto” option from fstab and put a mount command like sleep 30 && mount /mnt/myshare & in your /etc/rc.local (sleep ensures that your mount happens after 30 seconds when your STP switch port starts to forward).

Happy hacking!!!

Note: 30 seconds it’s just an indication, STP can take usually 30 to 50 seconds to respond to a topology change. If your switch uses  RSTP (Rapid Spanning Tree Protocol) it’s typically able to respond to changes within 3 × Hello times (default: 3 times 2 seconds).

Debian Squeeze/Wheezy (6.0/7.0) and Xerox Unified Driver v. 7-15

Xerox Unified Driver works very well and you can install it without any problem from console or by the X (gui) wizard. At the moment it doesn’t support officialy Debian >=6.0 (it supports old Debian 3.1, 4.0, 5.0 32/64bit) so this isn’t a real bug in fact but it can be a problem if you use squeeze/wheezy.

insserv: warning: script 'K07smfpd' missing LSB tags and overrides
insserv: warning: script 'smfpd' missing LSB tags and overrides
insserv: There is a loop at service rmnologin if started
insserv: There is a loop between service rmnologin and mountnfs if started
insserv:  loop involving service mountnfs at depth 6
insserv:  loop involving service nfs-common at depth 5
insserv: There is a loop between service rmnologin and checkroot if started
insserv:  loop involving service checkroot at depth 3
insserv:  loop involving service mountdevsubfs at depth 2
insserv: Starting smfpd depends on rmnologin and therefore on system facility `$all' which can not be true!
(...)
insserv: Starting smfpd depends on rmnologin and therefore on system facility `$all' which can not be true!
insserv: Max recursions depth 99 reached
insserv:  loop involving service portmap at depth 4
insserv: There is a loop between service rmnologin and pcmcia if started
insserv:  loop involving service pcmcia at depth 2
insserv:  loop involving service udev at depth 1
insserv:  loop involving service module-init-tools at depth 4
insserv:  loop involving service mountnfs-bootclean at depth 8
insserv:  loop involving service exim4 at depth 8
insserv: There is a loop between service rmnologin and mountoverflowtmp if started
insserv:  loop involving service mountoverflowtmp at depth 3
insserv:  loop involving service mountall-bootclean at depth 2
insserv:  loop involving service avahi at depth 8
insserv: There is a loop between service rmnologin and ifupdown if started
insserv:  loop involving service ifupdown at depth 5
insserv:  loop involving service ifupdown-clean at depth 4
insserv:  loop involving service fuse at depth 11
insserv:  loop involving service networking at depth 7
insserv:  loop involving service mountall at depth 5
insserv:  loop involving service checkfs at depth 4
insserv: There is a loop between service rmnologin and ifupdown-clean if started
insserv: There is a loop at service smfpd if started
insserv:  loop involving service smfpd at depth 1
insserv:  loop involving service hwclockfirst at depth 4
insserv:  loop involving service hostname at depth 5
insserv: There is a loop between service rmnologin and mountall if started
insserv:  loop involving service mtab at depth 7
insserv: exiting now without changing boot order!
update-rc.d: error: insserv rejected the script header

To solve this problem you can:

  1. uninstall Xerox Unified Driver, update your system or install your software and then reinstall the Xerox driver
  2. “If your MFP device is not connected to LPT port, you can safely disable execution of this script” so remove /etc/init.d/smfpd (and its symlinks) from your init.d and rc.X folders
  3. Recommended: “LSBize” the init script to fix the problem (http://wiki.debian.org/LSBInitScripts). Add these lines to your /etc/init.d/smfpd (at EOF is ok because position is not important, so you can also use a combo of cat /patch >> /etc/init.d/smfpd if you want to fix it automatically) create a new file named smfpd in /usr/share/insserv/overrides/ with the following content:
    ### BEGIN INIT INFO
    # Provides:          smfpd
    # Required-Start:    $syslog
    # Required-Stop:     $syslog
    # Default-Start:     2 3 4 5
    # Default-Stop:      0 1 6
    # Short-Description: Xerox Unified Linux Driver LPT Support daemon
    # Description:       Enable support for LPT port service provided by Xerox Unified Linux Driver package.
    ### END INIT INFO

And just request support for Debian -6.0- 7.0 😉

update 2011-08-12: there is another minor bug in install.sh at line 1187 because –reload_rules is wrong, change it to –reload-rules
update 2013-06-14: mmmm, Xerox didn’t fix, so I’ve updated my post for Debian Wheezy 7.0
update 2013-07-02: thanks to darac’s comment I’ve learned a better way to do the work: put an override file in /usr/share/insserv/overrides/ as indicated by insserv man page

How-to Install Debian 6.0 (squeeze) Linux on HP 620 notebook

HP 620. It’s a great notebook for business. Hp620 is not the best about performance but it’s fast – really fast! with linux – and it’s ok with it’s easy and clean design. Its hardware is very good (almost everything is manufactured by Intel) and it’s cheap!

Above all, you can also buy it pre-installed with Linux (Suse Enterprise Desktop)!

If you want Debian just install Debian 6.0 (Squeeze) with Desktop System, Notebook and Basic System files.
After installation run these commands from shell (as root).

cd ~
mv /etc/apt/sources.list /etc/apt/sources.list.original
touch /etc/apt/sources.list
echo "deb http://ftp.it.debian.org/debian/ squeeze main contrib non-free" >> /etc/apt/sources.list
echo "deb-src http://ftp.it.debian.org/debian/ squeeze main contrib non-free" >> /etc/apt/sources.list
echo "deb http://ftp.it.debian.org/debian/ squeeze-updates main contrib non-free" >> /etc/apt/sources.list
echo "deb-src http://ftp.it.debian.org/debian/ squeeze-updates main contrib non-free" >> /etc/apt/sources.list
echo "deb http://security.debian.org/ squeeze/updates main contrib" >> /etc/apt/sources.list
echo "deb-src http://security.debian.org/ squeeze/updates main contrib" >> /etc/apt/sources.list
apt-get update
apt-get install flashplugin-nonfree icedove icedove-l10n-it ntp wireless-tools
apt-get --yes install linux-headers-2.6-686 build-essential module-assistant pulseaudio firmware-ralink
m-a update && m-a prepare
m-a a-i -t alsa-source
echo "acpiphp" >> /etc/modules
echo "ata_piix" >> /etc/modules
echo "cfg80211" >> /etc/modules
echo "iTCO_vendor_support" >> /etc/modules
echo "iTCO_wdt" >> /etc/modules
echo "pci_hotplug" >> /etc/modules
echo "rt3090sta" >> /etc/modules
echo "snd_mixer_oss" >> /etc/modules
echo "snd_pcm_oss" >> /etc/modules
echo "speedstep_lib" >> /etc/modules
echo "tpm" >> /etc/modules
echo "tpm_bios" >> /etc/modules
echo "tpm_tis" >> /etc/modules
echo "auto lo" > /etc/network/interfaces
echo "iface lo inet loopback" >> /etc/network/interfaces
echo "allow-hotplug eth0" >> /etc/network/interfaces

Now you have a perfect business laptop with browser (firefox/iceweasel), email client (thunderbird/icedove) and openoffice (and whatever you want to install).

All hardware now works very well without any problem.
Enjoy.

Incendio Sala UPS Aruba DOWN – Tempi di ripristino

Purtroppo la sala batterie di Aruba ha subito un principio di incendio e per questo tutti i servizi sono stati disattivati (Aruba e tutti i servizi collegati sono down). Secondo le loggate del mio server in housing su Aruba il down è iniziato alle 4.39 circa.

Poco fa sono riuscito a mettermi in contatto con il team di Aruba che attualmente risulta down.
Hanno confermato i primi interventi tecnici specializzati nel primo pomeriggio ed il conseguente graduale riavvio dei servizi nella nottata.

Il canale “ufficiale” di Aruba per avere informazioni aggiornate:

http://twitter.com/#!/arubait

Wikileaks and Cloud computing

What do you think about WikiLeaks doesn’t matter.
Think about the future. Your browser will be on a cloud.
The sun will shine suddenly.

How-to convert an existing partition/disk in an image to virtualize it in linux qemu/kvm

Hi!
Yes, you can.
You can virtualize an existing windows/linux installation with qemu / kvm.
This isn’t a real news but here I write a very basic/fast tutorial.

Basically you have to choose between:

  • use your linux/windows hd by moving it to your kvm host server
  • convert your linux/windows to a kvm image (qcow2, row, ecc.) and run it on your kvm host server

Important note – 25/01/2011
Please note that when you virtualize a disk inside kvm, as we do in this tutorial, its OS find a new hardware configuration (fake/virtualized by kvm). This maybe a problem just because some operating systems (especially windows) doesn’t like that situation and panics with blue screen at boot & co©. To avoid this kind of problems make sure to uninstall specific drivers from your source disk OS before generate your image (especially windows!). Especially for windows (!!!) ensure to uninstall the IDE/SATA controller driver and replace it with general/universal driver. I have no problems like that to report for Linux os.

For the first choice the answer is simple: just use kvm’s -hda (or -drive) argument and point to your disk. Kvm and qemu can use a real hd without problems.

In order to convert an existing partition/disk to a qemu / kvm image (second choice) you have to follow these steps:

  • shutdown your linux/windows machine and reboot from cd with a live distro (use SystemRescueCd to follow this tutorial)
  • create a new folder to mount the image destination folder, it can’t be on our machine disk… we have to create our new kvm/qemu image from it!
    mkdir /mnt/image-dest
  • mount a remote samba partition with cifs in your destination folder (mount -t cifs //192.168.1.x/your-samba-share-name) or mount an usb drive, or a nfs partition, or your smartphone, that doesn’t matter! 🙂 You only need to transfer your windows/linux disk image to your qemu / kvm host.
  • umount your windows/linux partitions if one is mounted for any reason
  • create your disk raw image with dd from winows/linux disk; note that you have to copy ALL disk, not only a single partition. For example, if you have 3 partitions (/boot hda1 / hda2 and swap hda3) you have to use dd on your disk device, not on partitions. Wrong: dd if=/dev/hda1. Correct: dd if=/dev/hda. Please note that with dd you can kill your cat, burn your sister and immediately join Forza Italia. Use it with care!
    dd if=/dev/hda of=/mnt/image-dest/kvm-image.raw
  • fire up your kvm raw image: this may vary based on your source machine hardware configuration, if you have problems with missing partitions errors on your guest os boot phase just look at kvm / qemu -drive options (man kvm);
    kvm -drive file=/path/to/your/kvm-image.raw format=raw media=disk
  • optionally convert your raw image to other formats with kvm-img

TIP: Note that this process can be slow if your source disk is very large and may waste disk space on your destination host, especially if your source disk is large but only a small part of it’s used. In that case you can use the qemu-img tool to convert the physical disk to a kvm/qemu image: this tool allows you to use the -S/sparse option to convert (http://en.wikipedia.org/wiki/Sparse_file). The image of your source disk converted with the sparse option will be as big as the source used space.
Of course before convert with the sparse option you need a way to set to zero all the unused bytes in your source disk: for ext2, ext3 and ext4 partitions you can use the zerofree tool (https://packages.debian.org/wheezy/zerofree) or you can try another way (an easy way that works with all filesystems may be create a zero filled file on your source disk that fills its free space). Another way may be use a tool like parted to reduce partitions size and then copy only used disk space with a combo of dd’s ibs and count (untested).

Happy virtualization to you 🙂

UPDATE 2014-07-27: added a tip about big source disks or source disks with a lot of free space

debian, grub-pc and /usr/sbin/grub-probe: error: unknown filesystem.

apt-get dist-upgrade and… BOOM!!!

Configurazione di grub-pc (1.98~20100128-1)…
/usr/sbin/grub-probe: error: unknown filesystem.
Auto-detection of a filesystem module failed.
Please specify the module with the option `–modules’ explicitly.
Generating grub.cfg …
/usr/sbin/grub-probe: error: unknown filesystem.
dpkg: errore nell’elaborare grub-pc (–configure):

Not good.

Bug Report and my solution, downgrade to 1.98~20100115-1:

apt-get install grub-common= grub-pc=1.98~20100115-1 grub2=1.98~20100115-1
(…)
Installation finished. No error reported.

Have a nice day!