+3  A: 

You need to make sure to specify the right calling convention in your function pointer prototype ('CALLBACK' might be the wrong choice).

Alexander Gessler
can you explain that in more detail?
Nick Brooks
+2  A: 

The calling code uses the calling convention not matching that of the function being called. See this very similar question. You need to open the header defining that function (should come with the library you try to use), look the convention up and change your function pointer declaartion accordingly.

sharptooth
Using WINAPI didn't help :(
Nick Brooks
That's not necessarily WINAPI, that might be anything. You need to look into the library documentation to find which convention to use.
sharptooth
There is no documentation , but does knowing the syntax for C# function help?
Nick Brooks
Didn't notice "C#" until now. Not even sure that calling a function from a managed DLL will work without using some interop. Have you searched for how to call managed code from unmanaged code? I can't help you here, only can say that we exposed the C# library to COM for the same purpose and used it through COM.
sharptooth
that is not what i meant. the dll is written in c++ but i know how to use it from c# : [DllImport("MiniFMOD.dll")] public static extern int SongLoadFromFile(string name);i don't have any headers or documentation for the dll
Nick Brooks
using __cdecl doesn't return any errors but songHandle comes up as 0
Nick Brooks
I can't give a "use this and it'll work" answer - I just don't know. I suggest that you look into http://msdn.microsoft.com/en-us/library/k2b2ssfy.aspx and just try all available conventions - there're no many of them. However MSDN says that the default calling convention for [DllImport] is WinAPI which you say you tried without luck. Maybe the reason is that WINAPI symbol is defined some wrong way because of some peculiarities in your build environment. So I suggest that you try "raw" convention names - __stdcall, __cdecl, etc. This will not take a lot of time to test.
sharptooth
Tried all of them - function either returns 0 or an error
Nick Brooks
Are you sure 0 isn't a valid 'song handle' (it's an opaque handle, so why shouldn't 0 be an allowed value)? __cdecl seems reasonable to me.
Alexander Gessler
+2  A: 

I think the other people are misunderstanding the question. It seems to me that minifmod.dll is a native library that exports a function named SongLoadFromFile. The existing code that calls this is managed code (C#) that uses DllImport to call the function in the native DLL. From what little information I could gather by a few Google searches, it looks as though it should be declared as follows:

typedef int (__cdecl * SongLoadFromFileT)(const char*);

Importantly, it is __cdecl calling convention and it takes an ANSI string instead of a Unicode string.

As an aside, I find it strange that I can't find ANYTHING on minifmod.dll other than a few forum posts on a Russian website and some SO questions from this guy. The only "legitimate" information I can find on minifmod is a small static library with similar functionality. I wonder if minifmod.dll is some kind of commercialized version of the static library; at least that would explain why there is not much public documentation about it.

Ah, I found it; it is a Delph port of minifmod (http://www.cobans.net/minifmod.php).

Luke
The example you gave me doesn't trigger any errors , BUT the songHandle is still 0 meaning that file doesn't exist or that something went wrong while calling the function
Nick Brooks
Are you passing an ANSI string? Your sample code is not. Also, you are not escaping the backslash character; the path should be "C:\\b.xm".If something goes wrong, it is customary for Win32 libraries to call SetLastError() with whatever error code happened; you can call GetLastError() to see if it gets set. Other than that, I'm not a psychic and can't help any further without documentation.You could always try running File Monitor to see what file it is actually trying to access (if any), or you could step into the assembly code if you are desperate.
Luke
OMFG , ESCAPING THE BACKSLASH WORKED. I LOVE YOU MAN!
Nick Brooks
that was the reason it kept returning 0 , because of that backslash thingie because it couldn't retrieve the file
Nick Brooks