I want to modify a C program to make some of the files it creates hidden in Windows. What Windows or (even better) POSIX API will set the hidden file attribute?
You can do it by calling SetFileAttributes and setting the FILE_ATTRIBUTE_HIDDEN flag. See http://msdn.microsoft.com/en-us/library/aa365535%28VS.85%29.aspx
This is not POSIX though. To create a 'hidden' file under a normal POSIX system like Linux, just start a filename with a dot (.).
Windows and UNIX-like systems have different views on what exactly is a hidden file. On UNIX-likes conventionally file names starting with a dot are considered "hidden". Windows file systems on the other hand have a "hidden" attribute for files.
So for POSIX you should probably just create your files with a starting dot in the file name.
On Windows you can use the SetFileAttributes function.
You are looking for the GetFileAttributesEx, GetFileAttributes and SetFileAttributes set of methods in the Win32 API.
Starting point of the documentation