+2  A: 

Answering my own question here. The problem was that I was extending Surface (which I really in the end didn't have a proper reason for) which apparently uses OpenGL for rendering and OpenGL and custom TTF fonts don't mix.

When I switched to extending just View, the font looks perfect.

TuomasR