views:

9613

answers:

8

I have an NFS-mounted directory on a Linux machine that has hung. I've tried to force an unmount, but it doesn't seem to work:

$ umount -f /mnt/data
$ umount2: Device or resource busy
$ umount: /mnt/data: device is busy

If I type "mount", it appears that the directory is no longer mounted, but it hangs if I do "ls /mnt/data", and if I try to remove the mountpoint, I get:

$ rmdir /mnt/data
rmdir: /mnt/data: Device or resource busy

Is there anything I can do other than reboot the machine?

+17  A: 

You might try a lazy unmount. umount -l

tessein
This is not implemented everywhere. I don't have it on FreeBSD, for instance.
Daniel Papasian
+4  A: 

Try running

lsof | grep /mnt/data

That should list any process that is accessing /mnt/data that would prevent it from being unmounted.

Ryan Ahearn
A: 

@Ryan

I tried lsof, but it hung as well. I ran strace lsof, and it ended like this:

alarm(0)                                = 0
rt_sigaction(SIGALRM, {SIG_DFL}, {0x40f680, [ALRM], SA_RESTORER|SA_RESTART, 0x3bfa430f30}, 8) = 0
close(5)                                = 0
close(6)                                = 0
rt_sigaction(SIGALRM, {0x40f680, [ALRM], SA_RESTORER|SA_RESTART, 0x3bfa430f30}, {SIG_DFL}, 8) = 0
alarm(5)                                = 0
wait4(-1,

I even tried the following to identify what process was accessing the directory:

find /proc -wholename '*/fd/*' -exec file {} \;  | grep "data"

Unfortunately, there were no symlinks to or inside the /mnt/data directory. tessein's solution did the trick, though.

lorin
+5  A: 

If the NFS server disappeared and you can't get it back online, one trick that I use is to add an alias to the interface with the IP of the NFS server (in this example, 192.0.2.55). In Linux the command for that is something roughly like:

ifconfig eth0:fakenfs 192.0.2.55 netmask 255.255.255.255

Where 192.0.2.55 is the IP of the NFS server that went away. You should then be able to ping the address, and you should also be able to unmount the filesystem (use unmount -f). You should then destroy the aliased interface so you no longer route traffic to the old NFS server to yourself.

On FreeBSD and similar operating systems, the command would be something like:

ifconfig em0 alias 192.0.2.55 netmask 255.255.255.255

And then to remove it:

ifconfig em0 delete 192.0.2.55

man ifconfig(8) for more!

Daniel Papasian
A combination of `ifconfig eth0:fakenfs ...' and `umount -f -l /my/mount/dir' solved the problem for me.
pts
A: 

Your NFS server disappeared.

Ideally your best bet is if the NFS server comes back.

If not, the "umount -f" should have done the trick. It doesn't ALWAYS work, but it often will.

If you happen to know what processes are USING the NFS filesystem, you could try killing those processes and then maybe an unmount would work.

Finally, I'd guess you need to reboot.

Also, DON'T soft-mount your NFS drives. You use hard-mounts to guarantee that they worked. That's necessary if you're doing writes.

A: 

After reboot also same error i am getting $ umount -f /mnt/data $ umount2: Device or resource busy $ umount: /mnt/data: device is busy

Regards Kathir

A: 

try to edit /etc/mtab then find the nfs server just give '#' in the begin of line

192.x.x.1:/data/backup/ /mnt nfs rw,addr=10.2.116.117 0 0

dani.jtk
This will still cause commands to hang if they access a directory where the old nfs mount was. This will only really clear up hanging df commands
Mike
A: 

Our NFS server doesn't exist. ?etc/fstab is clear but there is an entry in /etc/mtab. df command keeps hanging and the below trick (which was given by lorin above) worked for me.

Aliasing the nfs server's IP and umount -f worked for me but I needed to do "umount -f" multiple times.

Ram