I think this is the correct behavior although it is not what you might expect.
Consider that a RadioButton is just a CheckBox with some extended functionality to automatically to give that exclusive checking functionality. In the background it is still a checkbox though. See the hierarchy from MSDN:
System.Object
System.Web.UI.Control
System.Web.UI.WebControls.WebControl
System.Web.UI.WebControls.CheckBox
System.Web.UI.WebControls.RadioButton
The output has all items with the attribute checked="checked" output for the input of type="radio". Eg:
<input id="rad1" type="radio" name="Test" value="rad1" checked="checked" /><label for="rad1">1</label><br />
<input id="rad2" type="radio" name="Test" value="rad2" checked="checked" /><label for="rad2">2</label><br />
<input id="rad3" type="radio" name="Test" value="rad3" checked="checked" /><label for="rad3">3</label>
From the Checked property documenation:
Checked (inherited from CheckBox):
Gets or sets a value indicating
whether the CheckBox control is
checked.
So the Checked property acts just like the CheckBox version with no functionality included to look for other controls in the same group and remove them which makes sense since it is a singular control.