Trying to make an array of DirectX vertex with out knowing until run time what type they will be

Question

Bit of background for those who don't know DirectX. A vertex is not just an XYZ position, it can have other data in it as well. DirectX uses a system known as Flexible Vertex Format, FVF, to let you define what format you want your vertexs to be in. You define these by passing a number to DirectX that use bitwise or to build it up, eg (D3DFVF_XYZ | D3DFVF_DIFFUSE)means you are going to start using (from when you tell DirectX) vertexs that have an XYZ (three floats) and a RGB components (DWORD / unsigned long).

In order to pass your vertexs to the graphics card, you basicaly lock the memory in the graphics card where your buffer is, and use memcpy to transfer your array over.

Your array is an array of a struct you deffine your self, so in this case you would have made a struct like...

struct CUSTOMVERTEX {
    FLOAT X, Y, Z;
    DWORD COLOR;
};

You then make an array of type CUSTOMVERTEX and fill in the data fields.

I think my best appraoch is let my class build up an array of each component type, so an array of struct pos{ flaot x,y,z;}; an array of struct colour{ DWROD colour;}; etc.

But I will then need to merge these together so that I have an array structs like CUSTOMVERTEX.

Now, I think I have made a function that will merge to arrays together, but I am not sure if it is going to work as intended, here it is (currently missing the abilaty to actually return this 'interlaced' array)

void Utils::MergeArrays(char *ArrayA, char *ArrayB, int elementSizeA, int elementSizeB, int numElements)
{
    char *packedElements = (char*)malloc(numElements* (elementSizeA, elementSizeB));
    char *nextElement = packedElements;
    for(int i = 0; i < numElements; ++i)
    {
        memcpy(nextElement, (void*)ArrayA[i], elementSizeA);
        nextElement += elementSizeA;
        memcpy(nextElement, (void*)ArrayB[i], elementSizeB);
        nextElement += elementSizeB;
    }
}

when calling this function, you will pass in the two arrays you want merged, and size of the elements in each array and the number of elements in your array.

I was asking about this in chat for a while whilst SO was down. A few things to say.

I am dealing with fairly small data sets, like 100 tops, and this (in theory) is more of an initialisation task, so should only get done once, so a bit of time is ok by me.

My final array that I want to be able to use memcpy on to transfer into the graphics card needs to have no padding, it has to be contiguous data.

EDIT The combined array of vertex data will be transfered to the GPU, this is first done by requesting the GPU to set a void* to the start of the memory I have access to and requesting space the size of my customVertex * NumOfVertex. So if my mergeArray function does loose what the types are within it, that is ok, just a long as I get my single combined array to transfer in one block /EDIT

Finally, their is a dam good chance I am barking up the wrong tree with this, so their may well be a much simpler way to just not have this problem in the first place, but part of me has dug my heals in and wants to get this system working, so I would appreciate knowing how to get such a system to work (the interlacing arrays thing)

Thank you so much... I need to sooth my head now, so I look forward to hearing any ideas on the problem.

Answer 1

I don't know DirectX, but the exact same sort of concept exists in OpenGL, and in OpenGL you can specify the location and stride of each vertex attribute. You can have alternating attributes (like your first struct) or you scan store them in different blocks. In OpenGL you use glVertexPointer to set these things up. Considering that DirectX is ultimately running on the same hardware underneath, I suspect there's some way to do the same thing in DirectX, but I don't know what it is.

Some Googling with DirectX and glVertexPointer as keywords turns up SetFVF and SetVertexDeclaration

MSDN on SetFVF, gamedev discussion comparing them

Answer 2

Your solution looks fine. But if you want something a bit more C++ish, you could try something like this:

Edit My previous solution basically recreated something that already existed, std::pair. I don't know what I was thinking, here's the even more C++ish solution:

template<typename InIt_A, typename InIt_B, typename OutIt>
void MergeArrays(InIt_A ia, InIt_B ib, OutIt out, std::size_t size)
{
    for(std::size_t i=0; i<size; i++)
    {
        *out = make_pair(*ia,*ib);
        ++out;
        ++ia;
        ++ib;
    }
}

int main()
{
    pos p[100];
    color c[100];
    typedef pair<pos,color> CustomVertex;
    CustomVertex cv[100];

    MergeArrays(p,c,cv,100);
}

You shouldn't have to worry about padding, because all elements in a D3D vertex are either 32 bit floats, or 32 bit integers.

Edit

Here's a solution that might work. It will do all your mergings at once, and you don't need to worry about passing around the size:

// declare a different struct for each possible vertex element
struct Position { FLOAT x,y,z; };
struct Normal { FLOAT x,y,z; };
struct Diffuse { BYTE a,r,g,b; };
struct TextureCoordinates { FLOAT u,v; };
// etc...

// I'm not all too sure about all the different elements you can have in a vertex
// But you would want a parameter for each one in this function.  Any element that
// you didn't use, you would just pass in a null pointer.  Since it's properly
// typed, you won't be able to pass in an array of the wrong type without casting.
std::vector<char> MergeArrays(Position * ppos, Normal * pnorm, Diffuse * pdif, TextureCoordinates * ptex, int size)
{
    int element_size = 0;
    if(ppos) element_size += sizeof(Position);
    if(pnorm) element_size += sizeof(Normal);
    if(pdif) element_size += sizeof(Diffuse);
    if(ptex) element_size += sizeof(TextureCoordinates);

    vector<char> packed(element_size * size);

    vector<char>::iterator it = packed.begin();
    while(it != packed.end())
    {
        if(ppos)
        {
            it = std::copy_n(reinterpret_cast<char*>(ppos), sizeof(Position), it);
            ppos++;
        }

        if(pnorm)
        {
            it = std::copy_n(reinterpret_cast<char*>(pnorm), sizeof(Normal), it);
            pnorm++;
        }

        if(pdif)
        {
            it = std::copy_n(reinterpret_cast<char*>(pdif), sizeof(Diffuse), it);
            pdif++;
        }

        if(ptex)
        {
            it = std::copy_n(reinterpret_cast<char*>(ptex), sizeof(TextureCoordinates), it);
            ptex++;
        }

    }

    return packed;
}

// Testing it out.  We'll create an array of 10 each of some of the elements.
// We'll use Position, Normal, and Texture Coordinates.  We'll pass in a NULL
// for Diffuse.
int main() 
{
    Position p[10];
    Normal n[10];
    TextureCoordinates tc[10];

    // Fill in the arrays with dummy data that we can easily read.  In this
    // case, what we'll do is cast each array to a char*, and fill in each
    // successive element with an incrementing value.
    for(int i=0; i<10*sizeof(Position); i++)
    {
        reinterpret_cast<char*>(p)[i] = i;
    }

    for(int i=0; i<10*sizeof(Normal); i++)
    {
        reinterpret_cast<char*>(n)[i] = i;
    }

    for(int i=0; i<10*sizeof(TextureCoordinates); i++)
    {
        reinterpret_cast<char*>(tc)[i] = i;
    }

    vector<char> v = MergeArrays(p,n,NULL,tc,10);

    // Output the vector.  It should be interlaced:
    // Position-Normal-TexCoordinates-Position-Normal-TexCoordinates-etc...
    for_each(v.begin(), v.end(),
        [](const char & c) { cout << (int)c << endl; });
    cout << endl;
}

Answer 3

Altering your code, this should do it:

void* Utils::MergeArrays(char *ArrayA, char *ArrayB, int elementSizeA, int elementSizeB, int numElements)
{
    char *packedElements = (char*)malloc(numElements* (elementSizeA + elementSizeB));
    char *nextElement = packedElements;
    for(int i = 0; i < numElements; ++i)
    {
        memcpy(nextElement, ArrayA + i*elementSizeA, elementSizeA);
        nextElement += elementSizeA;
        memcpy(nextElement, ArrayB + i*elementSizeB, elementSizeB);
        nextElement += elementSizeB;
    }
    return packedElements;
}

Note that you probably want some code that merges all the attributes at once, rather than 2 at a time (think position+normal+texture coordinate+color+...). Also note that you can do that merging at the time you fill out your vertex buffer, so that you don't ever need to allocate packedElements.

Something like:

//pass the Locked buffer in as destArray
void Utils::MergeArrays(char* destArray, char **Arrays, int* elementSizes, int numArrays, int numElements)
{
    char* nextElement = destArray;
    for(int i = 0; i < numElements; ++i)
    {
        for (int array=0; array<numArrays; ++array)
        {
            int elementSize = elementSizes[array];
            memcpy(nextElement, Arrays[array] + i*elementSize, elementSize);
            nextElement += elementSize;
        }
    }
}

Answer 4

No, no, no. The FVF system has been deprecated for years and isn't even available in D3D10 or later. D3D9 uses the VertexElement system. Sample code:

D3DVERTEXELEMENT9 VertexColElements[] =
{
    {0, 0, D3DDECLTYPE_FLOAT3, D3DDECLMETHOD_DEFAULT, D3DDECLUSAGE_POSITION, 0},
    {0, 12, D3DDECLTYPE_D3DCOLOR, D3DDECLMETHOD_DEFAULT, D3DDECLUSAGE_COLOR, 0},
    D3DDECL_END(),
};

The FVF system has a number of fundamental flaws - for example, which order the bytes go in.

On top of that, if you want to make a runtime-variant vertex data format, then you will need to write a shader for every possible variant that you may want to have, and compile them all, and spend your life swapping them around. And, the effects on the final product would be insane - for example, how could you possibly write a competitive rendering engine if you decide to take out the lighting data you need to Phong shade?

The reality is that a runtime-variant vertex format is more than a tad insane.

However, I guess I'd better lend a hand. What you really need is a polymorphic function object and some plain memory- D3D takes void*s or somesuch so that's not a big deal. When you call the function object, it adds to the FVF declaration and copies data into the memory.

class FunctionBase {
public:
    virtual ~FunctionBase() {}
    virtual void Call(std::vector<std::vector<char>>& vertices, std::vector<D3DVERTEXELEMENT9>& vertexdecl, int& offset) = 0;
};
// Example implementation
class Position : public FunctionBase {
    virtual void Call(std::vector<std::vector<char>>& vertices, std::vector<D3DVERTEXELEMENT9>& vertexdecl, int& offset) {
        std::for_each(vertices.begin(), vertices.end(), [&](std::vector<char>& data) {
            float x[3] = {0};
            char* ptr = (char*)x;
            for(int i = 0; i < sizeof(x); i++) {
                data.push_back(ptr[i]);
            }
        }
        vertexdecl.push_back({0, offset, D3DDECLTYPE_FLOAT3, D3DDECLMETHOD_DEFAULT, D3DDECLUSAGE_POSITION, 0});
        offset += sizeof(x);
    }
};

std::vector<std::vector<char>> vertices;
std::vector<D3DVERTEXELEMENT9> vertexdecl;
vertices.resize(vertex_count);
std::vector<std::shared_ptr<FunctionBase>> functions;
// add to functions here
int offset = 0;
std::for_each(functions.begin(), functions.end(), [&](std::shared_ptr<FunctionBase>& ref) {
    ref->Call(vertices, vertexdecl, offset);
});
vertexdecl.push_back(D3DDECL_END());

Excuse my use of lambdas, I use a C++0x compiler.