#include <header_name>
: Standard include file: look in standard paths (system include paths setup for the compiler) first
#include "header_name"
: Look in current path first, then in include path (project specific lookup paths)
The benefit of using a header file is to provide others with the interface of your library, without the implementation. Java does not require it because java bytecode or jar is able to describe itself (reflexion). C code cannot (yet) do it.
In Java you would only need the jar and have the correct use statement. In C you will (mostly) need a header and a lib file (or header and dll).
The other reason is the way c code is compiled. The compiler compiles translation units (a c/cpp file with all included headers) and the linker in a second step links the whole stuff. Declarations must not be compiled, and this saves time and avoid code to be useless generated for each compilation unit which the linker would have to cleanup.
This is just a general idea, I am not a compiler specialist but should help a bit.