It looks like there are two culprits:
The mini-
JPanel
containing the two radio buttons has aFlowLayout
which defaults to adding 5 pixels of padding around each component.Doing
radioButton.setBorder(null)
eliminates another pixel's worth of space around the buttons. It also screws up the dotted line drawn around them when they have focus, though.