Isn't this odd because B extends A?
You have the right idea, but in the wrong direction. Let's consider an example that is easier to reason about:
class Animal {}
class Reptile : Animal {}
class Snake : Reptile {}
class Mammal : Animal {}
class Tiger : Mammal {}
class Giraffe : Mammal {}
delegate void D(Mammal m);
static void DoAnimal(Animal a) {}
static void DoMammal(Mammal m) {}
static void DoTiger(Tiger t) {}
D dm = DoMammal;
dm(new Tiger());
That's clearly legal. dm needs to be a method that takes a Mammal, and it is.
D dt = DoTiger;
dt(new Giraffe());
That's clearly got to be illegal. You cannot assign a method that takes a tiger to a delegate that takes a mammal, because a delegate that takes a mammal can take any mammal, not just a tiger. If this were legal then it would be possible to pass a giraffe to a method that takes a tiger.
What about this?
D da = DoAnimal;
da(new Giraffe());
That's fine. da is a delegate to a method that takes any mammal. A method that takes any animal clearly also takes any mammal. You can assign DoAnimal(Animal) to a delegate D(Mammal) because Mammal extends Animal. You see now how you got the direction of extension backwards?
Return types on the other hand work the way you think they do:
delegate Mammal F();
static Animal GetAnimal() {...}
static Mammal GetMammal() {...}
static Tiger GetTiger() {...}
F fm = GetMammal;
Mammal m = fm();
No problem there.
F ft = GetTiger;
Mammal t = ft();
No problem there; GetTiger returns a Tiger, so you can assign it to a delegate that requires that its target returns a mammal.
F fa = GetAnimal;
Mammal a = fa();
That's no good. GetAnimal might return a Snake, and now you have a variable typed as Mammal that contains a Snake. This has to be illegal.
This feature is called "covariance and contravariance of member group conversions" and it was introduced in C# 2.0. For more information on this topic see my article on it:
http://blogs.msdn.com/b/ericlippert/archive/2007/10/19/covariance-and-contravariance-in-c-part-three-member-group-conversion-variance.aspx