I guess that java.util.IdentityHashMap is what you're looking for (note, there's no IdentityHashSet). Lookup the API documentation:
This class implements the Map interface with a hash table, using reference-equality in place of object-equality when comparing keys (and values). In other words, in an IdentityHashMap, two keys k1 and k2 are considered equal if and only if (k1==k2). (In normal Map implementations (like HashMap) two keys k1 and k2 are considered equal if and only if (k1==null ? k2==null : k1.equals(k2)).)
This class is not a general-purpose Map implementation! While this class implements the Map interface, it intentionally violates Map's general contract, which mandates the use of the equals method when comparing objects. This class is designed for use only in the rare cases wherein reference-equality semantics are required.
edit: See Joachim Sauer's comment below, it's really easy to make a Set based on a certain Map. You'd need to do something like this:
Set<E> mySet = Collections.newSetFromMap(new IdentityHashMap<E, Boolean>());