var dupKeys = dict.GroupBy(innerD => innerD.Value)
.Where(mergedByValue => mergedByValue.Count() > 1)
.Select(mergedByValue => mergedByValue.OrderByDescending(m => m.Key).First().Key);
dict.Where(d => dupKeys.Contains(d.Key)).ToList()
.ForEach(d => dict.Remove(d.Key));
This assumes you want the last duplicate removed, where last is defined as the last ordinal string value.
If you want all duplicates removed, change your dupKeys to this:
var dupKeys = dict.GroupBy(innerD => innerD.Value)
.Where(mergedByValue => mergedByValue.Count() > 1).Dump()
.SelectMany(mergedByValue => mergedByValue.Select(m => m.Key));