I'd say the cleanest way to do what you are looking for is to use a Popup. The Popup class displays an element that floats above the rest of the elements on the screen, but is non-modal and can be configured to disappear when the user clicks away from it - perfect for your non-modal dialog. The Popup class has properties that allow you to control where it shows up relative to another control (in your case, the button you want the user to press to open the popup).
Here's an all-XAML example:
<Grid>
<ToggleButton HorizontalAlignment="Center" VerticalAlignment="Top"
x:Name="PopButton" Content="Pop"/>
<Popup Placement="Bottom" PlacementTarget="{Binding ElementName=PopButton}" StaysOpen="False"
IsOpen="{Binding ElementName=PopButton, Path=IsChecked, Mode=TwoWay}">
<Rectangle Height="100" Width="200" Fill="Blue"/>
</Popup>
</Grid>
You can also use commands or event handlers to open/close the popup from code.
The Placement and PlacementTarget properties set where the popup will appear, and which control it will appear relative to (there are other options that allow you to have it appear relative to its current position and relative to the mouse, too). Setting StaysOpen to False will have WPF automatically close the popup when the user clicks outside of it.
By default, a Popup has no style of it's own - it's just a container for floating content - so you'll have to style it to look like your window chrome/toolbar/etc. as appropriate.