YUV Conversion via a fragment shader...

Question

I have implemented a YUV to RGB conversion via a fragment shader written in Nvidia's shader language. (Y, U and V are stored in separate textures that are combined via multi texturing in my fragment shader). It works great under OpenGL, but under Direct3D I just can't get the output image to look right. I'm starting to suspect that Direct3D is somehow modifying the Y, U and V samples before I get a chance to do my YUV conversion thing. Does anyone know if Direct3D makes any modifications to the values stored in textures before the fragment shader is run and how to disable them>?

Answer 1

The only suggestion that comes to mind is that the textures are in an inappropriate format (low-precision or compressed).

Can you describe in what way the output looks wrong? Any chance of a right vs wrong screenshot?

Answer 2

I'll see if I can get screenshots to post tomorrow, until then I can say that the blues in the image are coming out as shades of green. It's actually pretty subtle. The texture formats are all 8 bit (I think L8)...

Answer 3

We figured it out. :) Basically the problem was that while our YUV to RGB equations were correct, we weren't properly sampling the V data! So no amount of futzing with the equations would have helped!

In the end, I would recommend the following strategy for anyone attempting to do this:

1) Set R, G, and B to the value from Y. You should get a grayscale image (as Y contains just luminance). 2) Next, set R, G, and B to U. You should get funny colors! 3) Finally set R, G, and B to V. Again, you should get funny colors.

Also, properly normalizing the values is critical. Check our fourcc.org for a discussion of proper YUV normalization.

Answer 4

Could you please be more precise? I cant get my yuv420p to rgb to work... please help! I have to admit i don't really understand shaders and yuv.

Here my shader which i stole together from somewhere :)

uniform sampler2D Ytex, Utex, Vtex;

void main(void) {

vec2 txCoord = vec2(gl_TexCoord[0].xy);

float y,u,v;

float r,g,b;

y = 1.1643 * (texture2D(Ytex, txCoord).r - 0.0625);

u = texture2D(Utex, txCoord).r - 0.5;

v = texture2D(Vtex, txCoord).r - 0.5;

r = y + 1.5958 * v;

g = y - 0.39173 * u - 0.81290 * v;

b = y + 2.017 * u;

gl_FragColor=vec4(r, g, b, 1.0);

}

unfortunately the result is just some interesting colored magenta/green video...