A lot depends on how you're using HasColumn. Are you calling it just once or twice, or repeatedly in a loop? Is the column likely to be there or is that completely unknown in advance?
Setting a row filter probably would do a table scan each time. (Also, in theory, GetSchemaTable() could generate an entirely new table with every call, which would be even more expensive -- I don't believe SqlDataReader does this, but at the IDataReader level, who knows?) But if you only call it once or twice I can't imagine this being that much of an issue (unless you have thousands of columns or something).
(I would, however, at least store the result of GetSchemaTable() in a local var within the method to avoid calling it twice in quick succession, if not cache it somewhere on the off chance that your particular IDataReader DOES regenerate it.)
If you know in advance that under normal circumstances the column you ask for will be present, the exception method is a bit more palatable (because the column not being there is, in fact, an exceptional case). Even if not, it might perform slightly better, but again unless you're calling it repeatedly you should ask yourself if performance is really that much of a concern.
And if you ARE calling it repeatedly, you probably should consider a different approach anyway, such as: call GetSchemaTable() once up front, loop through the table, and load the field names into a Dictionary or some other structure that is designed for fast lookups.