I do not think there is a Binding solution without a converter. But why not use one? You will often come across a requirement like this, so it makes sense to create some sort of MathConverter which takes some properties or parameters. Then you do not need to create a separate converter for each single requirement.
However, if you really do not want to use a converter, depending on your layout you might also use a star-sized grid where the AddButton is spread across two rows while the RemoveButton only occupies one row:
<Grid>
<Grid.RowDefinitions>
<RowDefinition Height="*"/>
<RowDefinition Height="*"/>
</Grid.RowDefinitions>
<Button Name="AddButton" Grid.Row="0" Grid.RowSpan="2" ... />
<Button Name="RemoveButton" Grid.Row="1" ... />
</Grid>
If you want the RemoveButton to be centered vertically, use this:
<Grid>
<Grid.RowDefinitions>
<RowDefinition Height="1*"/>
<RowDefinition Height="2*"/>
<RowDefinition Height="1*"/>
</Grid.RowDefinitions>
<Button Name="AddButton" Grid.Row="0" Grid.RowSpan="3" ... />
<Button Name="RemoveButton" Grid.Row="1" ... />
</Grid>
This way, the AddButton occupies three rows (4* in total) while the RemoveButton is in the center row (2*).
If it is not possible to add them to one shared Grid, you might make use of the Grid.IsSharedSizeScope
attached property. Details can be found here.