views:

219

answers:

2

I have made a simple webcam based application that detects the "edges of motion" so draws a texture that shows where the pixels of the current frame are significantly different to the previous frame. This is my code:

// LastTexture is a Texture2D of the previous frame.
// CurrentTexture is a Texture2D of the current frame.
// DifferenceTexture is another Texture2D.
// Variance is an int, default 100;

Color[] differenceData = new Color[CurrentTexture.Width * CurrentTexture.Height];
Color[] currentData = new Color[CurrentTexture.Width * CurrentTexture.Height];
Color[] lastData = new Color[LastTexture.Width * LastTexture.Height];

CurrentTexture.GetData<Color>(currentData);
LastTexture.GetData<Color>(lastData);

for (int i = 0; i < currentData.Length; i++)
{
    int sumCD = ColorSum(currentData[i]); // ColorSum is the same as c.R + c.B + c.G where c is a Color.
    int sumLD = ColorSum(lastData[i]);
    if ((sumCD > sumLD - Variance) && (sumCD < sumLD + Variance))
        differenceData[i] = new Color(0, 0, 0, 0); // If the current pixel is within the range of +/- the Variance (default: 100) variable, it has not significantly changes so is drawn black.
    else
        differenceData[i] = new Color(0, (byte)Math.Abs(sumCD - sumLD), 0); // This has changed significantly so is drawn a shade of green.
}

DifferenceTexture = new Texture2D(game1.GraphicsDevice, CurrentTexture.Width, CurrentTexture.Height);
DifferenceTexture.SetData<Color>(differenceData);

LastTexture = new Texture2D(game1.GraphicsDevice,CurrentTexture.Width, CurrentTexture.Height);
LastTexture.SetData<Color>(currentData);

Is there a way to offload this calculation to the GPU using shaders (it can go at about 25/26 fps using the above method, but this is a bit slow)? I have a basic understanding of how HLSL shaders work and don't expect a full solution, I just want to know if this would be possible and how to get the "difference" texture data from the shader and if this would actually be any faster.

Thanks in advance.

+2  A: 

You could sample two textures inside the pixel shader, then write the difference out as the colour value. If you set up a Render Target, the colour information you ouput from the shader will be stored in this texture instead of the framebuffer.

I don't know what sort of speed gain you'd expect to see, but that's how I'd do it.

*edit - Oh and I forgot to say, be aware of the sampling type you use, as it will affect the results. If you want your algorithm to directly translate to the GPU, use point sampling to start with.

Mark Simpson
Thanks, I'll try this out.
Callum Rogers
It seems to be nearly working except that Color in .NET has four values (0..255,0..255,0..255,0..255) that go to a float4 in HLSL (0..1,0..1,0..1,0..1) which seems to ruin everything.
Callum Rogers
Just like to clarify; this method works incredibly fast (finding the difference of two 640x480 textures in under 2ms), but rendering it is a complete nightmare (but, surprisingly, the shaders are not), so I have given up on this route, instead focusing on trying to dual thread the operation (one thread compares half the data, another the other half). I stumbled across a problem where it rendered the first frame and then stops doing anything else.
Callum Rogers
Regarding the shader route: None of these things should be a major problem; you will be able to work them out if you keep plugging away :)
Mark Simpson
+1  A: 

Regarding your comment above about deciding to use a dual thread approach to your problem, check out the .Net Parallel Extensions CTP from Microsoft. microsoft.com

If you're not planning on deploying to an XBox360, this library works great with XNA, and I've seen massive speed improvements in certain loops and iterations.

You would basically only have to change a couple lines of code, for example:

for (int i = 0; i < currentData.Length; i++)
{
    // ...
}

would change to:

Parallel.For(0, currentData.Length, delegate(int i)
{
   // ...
});

to automatically make each core in your processor help out with the number crunching. It's fast and excellent.

Good luck!

bufferz
I'll try this, looks much better than my current parallel attempt using arrays of delegates (ugh...)
Callum Rogers
OMG. This is incredible. Easy to setup and this is blazing fast; I've never seen anything like it. This has gone from 110ms (Original) to 60ms (My multithreading method) to 2ms (Shaders, but it was overly complex, unworkable and horribly inaccurate) to 15ms (Threading.For()). This now runs faster than 16ms required for 60fps and there is no lag. Thank you!
Callum Rogers