Simulate a device driver crash in linux. Have python reload it.

Question

I have a web camera running in Linux using the uvcvideo module. And I'm using a python application to access the web camera and display the image.

I want the python program to handle it if the web camera for some reason don't work anymore. Have tested with just unloading the module. Works fine if i just unload the module before I run the python code, but if force it to unload in use, I get the following feedback.

VIDIOC_DQBUF: Inappropriate ioctl for device

And if I kill the python code, and restart it the whole machine freezes.

The code I'm trying to run is

import pygame
import Image
from pygame.locals import *
import sys
import time, os

import opencv
from opencv import highgui 

camera = highgui.cvCreateCameraCapture(0)
fps = 10.0
pygame.init()
window = pygame.display.set_mode((640,480))
pygame.display.set_caption("WebCam Demo")
screen = pygame.display.get_surface()

while True:
 events = pygame.event.get()
 for event in events:
  if event.type == QUIT or event.type == KEYDOWN:
   sys.exit(0)
 while True:
  try:
   ima = highgui.cvQueryFrame(camera)
   im = opencv.adaptors.Ipl2PIL(ima)
   break;
  except TypeError:
   print 'No camera'
   os.system('sudo modprobe uvcvideo')
   time.sleep(1)
   camera = highgui.cvCreateCameraCapture(0)

 pg_img = pygame.image.frombuffer(im.tostring(), im.size, im.mode)
 screen.blit(pg_img, (0,0))
 pygame.display.flip()
 pygame.time.delay(int(1000 * 1.0/fps))

It's a modified version of http://www.jperla.com/blog/2007/09/26/capturing-frames-from-a-webcam-on-linux/ It's using openvc version 1.x and not 2.x.

Any idea on how to make this work?

Answer 1

Linux really doesn't like it when you try to remove a kernel driver while any processes are using it. I'm not convinced there's any good way for your userland application to do this (and having your app try to run 'sudo modprobe uvcvideo' is scary enough already).

Answer 2

Do you mean USB camera ? I don't know about forced unloading while module is in use, but this won't happen and is not a good simulation of camera not working anymore. Try to handle camera disconnection /reconnection gracefully first.

I don't know what you are trying to achieve when simulating driver crash, but you can't handle a driver crash, which can result in a oops or whatever, with user code. There is no defensive programming that can save you once kernel code is going wild.

Now, if an error (an error is different from a crash) occurs in driver code, then it should be returned to you, and all you can do is retry or exit. If your application is meant to be used by any UVC camera, then buy an USB camera which respect UVC, and play with it (disconnect / reconnect).

As for hardware failure, there is not much you can do, except perhaps setting a timeout. What you can do within your code is, if you discover a specific problem with the driver, is avoiding to trigger this specific problem. For instance, if you know changing from resolution x to resolution y leads to a freeze camera, or a driver oops, then avoid it.

But I would not spend much time trying to handle hypothetical crash you don't know anything about. Instead, you should try to exercise error code path. For example, what happens if your system is low on memory ? Or if your system load is such that your app can't keep up whith the incoming frame.

Answer 3

The reason your code now is crashing is because when the driver crashes, the device special files representing your hardware disappear. Your code still has open file handles to those devices. Depending on what exactly your code is doing behind the scenes, it is likely trying to issue an IOCTL to a now-invalid file handle, a use case that is typically not handled well by library code because it should only happen in an event like this with some kind of kernel-land fault that user-land code can't do anything about anyway.

Dealing with the camera if it stops working is completely different than dealing with the driver crashing. A malfunctioning camera should never take down a (correctly-written) driver. If the driver goes down, there's not much that your userland code will be able to do about it. Nor should it need to. If the driver crashes, that's the driver writers' problem, not yours. If you have a driver that is crashing on you often enough that you're tempted to try and handle it, then I would go with a different driver or try fixing the one you are using. No amount of application code is going to fix a faulty driver.

Don't forget that your code isn't the only code using the driver. Internal kernel processes or other applications may be using the driver as well. If something else is using the driver when you pull it, you can cause that other code to hang (beyond your control) and potentially take the whole system down.

Now if your webcam hardware has a problem, the driver gracefully should give you a message or an error of some kind that your application code can detect and act on, while doing its own work to get the camera working again. Failing hardware should not pose a burden on application code; let the driver do its job and it will bring the camera back online if possible. If it is unable to do so, then either the camera is in an unrecoverable state or the driver has room for improvement (if that is the case, making an offer to the driver's developers to test their code on your hardware can sometimes be a fast way of getting better driver support for your device).

Instead of trying to tear out the driver while it is running, I would concentrate on having code to handle all the possible error states that the driver can return for your device.