Do you really mean C# in the tags? HashMap is a Java type. Also it generally has two type parameters rather than one...
In C#, generic classes are always invariant. Some interfaces will be variant in C# 4, but very few (only those which either only use the type parameter in an output positiion, e.g. IEnumerable<T>, or only use the type parameter in an input position, e.g. IComparable<T>).
If you can provide more precise information about your situation, we'll probably be able to help come up with a simple solution - particularly if you can use LINQ with its Cast<T>() method.
EDIT: Okay, with HashSet<T>:
HashSet<BaseType> baseSet = ...;
var derivedSet = new HashSet<DerivedType>(baseSet.Cast<DerivedType>());
Note that even with C# 4 this would be necessary because the compiler doesn't know that every value in baseSet is an instance of DerivedType - there has to be an execution-time check. The reverse (creating a HashSet<BaseType> from a HashSet<DerivedType>) would work in C# 4.
FURTHER EDIT: If you just want to use UnionWith, that doesn't require a HashSet<DerivedType> - it requires an IEnumerable<DerivedType>. I suggest you do:
HashSet<BaseType> baseSet = ...;
HashSet<DerivedType> derivedSet = ...;
derivedSet.UnionWith(baseSet.Cast<DerivedType>());