What do I need to implement a basic map framework?

Question

I am doing iPhone development and what I want is a custom map framework (not route-me, not Google Maps). So far I found CATiledLayer and UISCrollView are helpful. But I am still not clear about the stuff(any kind of view, or anything from CoreAnimation framework, or anything else?) which would be perfectly fit this.

So far I've just made a sample which could display tiled images, which loads remote images asynchronously, but not any Geo related things.

I need some guidance of making a custom map framework from scratch. Sorry if this question is somewhat subjective. Thank you