Object files tend to get large because of large amounts of duplicated code and symbols placed into multiple object files. Typically this is caused by copies of inline functions and instantiated template code.
The resulting executable is much, much smaller than the sum of all the object files as duplicated weak symbols and unreferenced symbols can be discarded.
Personally, I wouldn't worry about object file size, but if you want to reduce it then minimize the use of inline functions and consider keeping template code in separate files from the other code and using explicit instantiation to generate the required instantiations in only one place.
Example
Suppose that you have this in a header file.
template< class T >
inline void F( T* t )
{
// ... some code
}
If you use F for a handful of types, then every translation unit will generate copies of the function for the types that it uses.
If you replace it with just a declaration:
template< class T > void F( T* t );
then you can generate just the instantations that you need in a separate source file:
template< class T > void F( T* t )
{
// definition
}
template void F< int >( int* );
template void F< char >( char* );
template void F< MyType >( MyType* );
Of course, you now have to manual manage which types you need a template instantiation for.