Here is some guide from Oracle Java:
Note that the ordering maintained by a
set (whether or not an explicit
comparator is provided) must be
consistent with equals if it is to
correctly implement the Set interface.
(See Comparable or Comparator for a
precise definition of consistent with
equals.) This is so because the Set
interface is defined in terms of the
equals operation, but a TreeSet
instance performs all key comparisons
using its compareTo (or compare)
method, so two keys that are deemed
equal by this method are, from the
standpoint of the set, equal. The
behavior of a set is well-defined even
if its ordering is inconsistent with
equals; it just fails to obey the
general contract of the Set interface.
I think in terms of technical, no, you don't have any problems. But, in terms of coding, readability and maintainability, you have to be careful, because other people can misuse or misunderstand what you are doing