I think you might be looking for window.open()
instead of window.showModalDialog()
Also, you aren't correctly concatenating the form field value into the url parameter of the method. Try the code below.
<asp:ImageButton ID="ImageButton2" runat="server"
ImageUrl="images/icon_edit_moderator.gif"
OnClientClick="window.open('search_staffM.aspx?id='+ document.forms[0].<%=Marketer_Staff_ID.ClientID %> +'&id2='+ document.forms[0].<%=Marketer_Staffname.clientID%>,'Search','width=550,height=170,left=150,top=200,scrollbars=1,toolbar=no,status=1')" /></td>
For a different, perhaps cleaner, approach I would suggest creating a separate javascript function and then use the OnClientClick to call it.
<script type='text/javascript'>
function openStaffDetails() {
var url = "search_staffM.aspx?id=" + document.forms[0].<%=Marketer_Staff_ID.ClientID %> + "&id2=" + document.forms[0].<%=Marketer_Staffname.clientID%>;
window.open(url, 'Search','width=550,height=170,left=150,top=200,scrollbars=1,toolbar=no,status=1');
return false;
}
</script>
<asp:ImageButton ID="ImageButton2" runat="server"
ImageUrl="images/icon_edit_moderator.gif"
OnClientClick="return openStaffDetails();" /></td>