Weird vertexshader/pixelshader glitch

Question

i've got a little problem with my water effect

as you can see here, it doesn't show up the right way. another screen with a diffrent texture applied shows the error in the transform something more clearly my HLSL code:

V2P vs(float4 inPos : POSITION, float2 inTex: TEXCOORD)
{    
    V2P Output = (V2P)0;

    float4x4 viewproj = mul (matView, matProjection);
    float4x4 worldviewproj = mul (matWorld,viewproj);

    float4x4 reflviewproj = mul (matRLView, matProjection);
    float4x4 reflworldviewproj = mul (matWorld, reflviewproj);

    Output.Position = mul(inPos, worldviewproj);
    Output.RLMapTex = mul(inPos, reflworldviewproj);

    return Output;
}
P2F ps(V2P PSIn)
{
    P2F Output = (P2F)0;        

    float2 ProjectedTexCoords;
    ProjectedTexCoords.x =   PSIn.RLMapTex.x / PSIn.RLMapTex.w /2.0f + 0.5f;
    ProjectedTexCoords.y =  -PSIn.RLMapTex.y / PSIn.RLMapTex.w /2.0f + 0.5f;    

    float2 ProjectedRefCoords;
    ProjectedRefCoords.x = ( PSIn.Position.x / PSIn.Position.w) /2.0f + 0.5f;
    ProjectedRefCoords.y = (-PSIn.Position.y / PSIn.Position.w) /2.0f + 0.5f; 

    Output.Color = tex2D(samRLMap, ProjectedTexCoords);    

    return Output;
}

the reflection map is rendered on a render target while flipping the y value of the eye along the waterheight. (and with up vector 0,-1,0)

so, my question: what could be the cuase of this?

Answer 1

I'm not clear on why you are changing x. Doesn't it stay the same as y is flipped? As in

    float2 ProjectedTexCoords;
    ProjectedTexCoords.x =   PSIn.RLMapTex.x / PSIn.RLMapTex.w;
    ProjectedTexCoords.y =  -PSIn.RLMapTex.y / PSIn.RLMapTex.w /2.0f + 0.5f;

Answer 2

Looks like a texture that is repeating its edge pixels. In other words, you might be doing a texture lookup beyond the texture boundaries. Are you sure your reflection map is big enough?

Maybe try setting the output colour to red if the texture coordinates are out of range? (I don't speak HLSL so I don't know the syntax for this, but I'm sure it is possible.)

Or enlarge the reflection map?

These kinds of issues can be hard to debug even if you can see the full source code, so this is more of a suggestion where to look, not an actual answer. My attempt at psychic debugging.

Answer 3

I guess i found it, the matrix i used for the reflected view, is wrong. When i use the standard view, it works fine