So you want:
User u1 = new User("Bill");
session.save(u1);
User u2 = new User("Bill");
session.saveOrUpdate(u2);
to notice that u1 and u2 are the same Bill and only store it once? Hibernate has no way of knowing that the Bills are the same person. It only looks at the id of User u2, sees that it's not set, concludes that u2 should be inserted, tries that and reports an exception.
SaveOrUpdate persists both a completely new object and a loaded object that's currently attached to the session. So this works (assuming you have a findByName method somewhere and there's another attribute, let's say favoriteColor):
User u1 = new User("Bill");
u1.setFavoriteColor("blue");
session.saveOrUpdate(u1);
User u2 = findByName("Joe");
u2.setFavoriteColor("red");
session.saveOrUpdate(u2);
But that's not what you want, right?
Your problem is made worse by your entity having a surrogate primary key and, separately, a business key (enforced via a uniqueness constraint). If you didn't have an id field and only name as primary key, you could use merge() (for a discussion of saveOrUpdate vs merge, look here):
User u1 = new User("Bill");
u1.setFavoriteColor("blue");
session.save(u1);
User u2 = new User("Bill");
u2.setFavoriteColor("red");
u2 = session.merge(u2);
But you don't have that, you need to enforce both the PK constraint on id and the uniqueness on name. So you'll need some variation of merge that does that. Very roughly, you'd like something along these lines:
public User mergeByName(User user) {
User merged;
User candidate = findByName(user.getName());
if (candidate != null) {
user.setId(candidate.getId());
merged = session.merge(user);
} else {
session.save(user);
merged = user;
}
return merged;
}