You need to understand the difference between server-side code and client-side code. Your code is written in C# which runs on the server. Here is what I assume is happening:
a. Your page renders for the first time and the user does not choose the relevant radiobutton. He/she then submits the page (possibly via the LinkButton).
b. Your page submits and the code you pasted runs on the server. You check if the radiobutton is checked. If not, you add an attribute to a LinkButton so that when the LinkButton is clicked, it will raise an alert.
c. Your page renders after postback with the new attribute added to the LinkButton.
d. On the click of the LinkButton, you get an alert with the message. Since you have set it to return false, the page will not submit again and will keep on showing you the alert.
Do you see what's happening here? Your alert condition needs to be checked on the client itself. The snippet provided by @Phoenix should be a good starting point.