If this is a homework assignment and you really are only restricted to the standard C library, you to be looking at the official PNG file format specification: http://www.w3.org/TR/PNG/. However, are you sure you really need to be decoding the PNG file? If all you need to do is display it somehow, you're headed on the wrong path.
It will be rather complex and time consuming to write a decoder for any general PNG file, but not too bad for simple ones. In fact, because the PNG format allows for pieces of it to be compressed, to do it with only standard C libraries would require you to implement gzip decompress (a reasonable homework assignment for a mid-level undergrad course, but my guess is that you would have spent a lot of discussing compression algoirthms before this was assigned to you)
However, it isn't terribly difficult if you restrict yourself to non-compressed, non-interlaced PNG files. I wrote a decoder once in Python that handled only the easy cases in a couple of hours, so I'm sure it'll be doable in C.