HLSL Shader to Subtract Background Image

Question

I am trying to get an HLSL Pixel Shader for Silverlight to work to subtract the background image from a video image. Can anyone suggest a more sophisticated algorithm than I am using because my algorithm isn't doing it correctly?

float Tolerance : register(C1);
SamplerState  ImageSampler : register(S0);
SamplerState  BackgroundSampler : register(S1);

struct VS_INPUT
{
    float4 Position : POSITION;
    float4 Diffuse  : COLOR0;
    float2 UV0      : TEXCOORD0;
    float2 UV1      : TEXCOORD1;
};

struct VS_OUTPUT
{
    float4 Position  : POSITION;
    float4 Color     : COLOR0;
    float2 UV        : TEXCOORD0;
};


float4 PS( VS_OUTPUT input ) : SV_Target
{
    float4 color = tex2D( ImageSampler, input.UV );
    float4 background = tex2D( BackgroundSampler, input.UV);

    if (abs(background.r - color.r) <= Tolerance && 
                  abs(background.g - color.g) <= Tolerance && 
                  abs(background.b - color.b) <= Tolerance)
    {
      color.rgba = 0;
    }

   return color;

}

To see an example of this, you need a computer with a webcam:

1. Go to the page http://xmldocs.net/alphavideo/background.html
2. Press [Start Recording].
3. Move your body out of the the scene and press [Capture Background].
4. Then move your body back into the scene and use the slider to adjust the Toleance value to the shader.

Answer 1

EDIT

Single pixel isn't useful for such task, because of noise. So algorithm essence should be to measure similarity between pixel blocks. Recipe pseudo-code (based on correlation measurement):

Divide image into N x M grid
For each N,M cell in grid:
   correlation = correlation_between(signal_pixels_of(N,M),
                                     background_pixels_of(N,M)
                                    );
   if (correlation > threshold)
      show_background_cell(N,M)
   else
      show_signal_cell(N,M)

This is sequential pseudo code, but it could be easily converted to HLSL shader. Simply each pixel detects to which pixel block it belongs, and after that measures correlation between corresponding blocks. And based on that correlation shows or hides current pixel.

Try this approach, Good Luck !