You can do this by customizing the existing Control Template for a ListBox Item. The easy way to do this is to fire up Expression Blend, right click a ListBoxItem, go to Edit Control Parts (Template) and select Edit a Copy... then customize the Fill color of the fillColor and fillColor2 rectangles as required.
The Xaml below sets the ListBoxItem mouse-over color to be transparent and the selected color to be bright green but you can customize this to your needs:
<UserControl x:Class="SilverlightApplication2.Page"
xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
xmlns:vsm="clr-namespace:System.Windows;assembly=System.Windows"
Width="400" Height="300" Background="#FF000000">
<UserControl.Resources>
<Style x:Key="ListBoxItemStyleTransparent" TargetType="ListBoxItem">
<Setter Property="Padding" Value="3"/>
<Setter Property="HorizontalContentAlignment" Value="Left"/>
<Setter Property="VerticalContentAlignment" Value="Top"/>
<Setter Property="Background" Value="Transparent"/>
<Setter Property="BorderThickness" Value="1"/>
<Setter Property="TabNavigation" Value="Local"/>
<Setter Property="Template">
<Setter.Value>
<ControlTemplate TargetType="ListBoxItem">
<Grid Background="{TemplateBinding Background}">
<vsm:VisualStateManager.VisualStateGroups>
<vsm:VisualStateGroup x:Name="CommonStates">
<vsm:VisualState x:Name="Normal"/>
<vsm:VisualState x:Name="MouseOver">
<Storyboard>
<DoubleAnimationUsingKeyFrames Storyboard.TargetName="fillColor" Storyboard.TargetProperty="Opacity">
<SplineDoubleKeyFrame KeyTime="0" Value=".35"/>
</DoubleAnimationUsingKeyFrames>
</Storyboard>
</vsm:VisualState>
<vsm:VisualState x:Name="Disabled">
<Storyboard>
<DoubleAnimationUsingKeyFrames Storyboard.TargetName="contentPresenter" Storyboard.TargetProperty="Opacity">
<SplineDoubleKeyFrame KeyTime="0" Value=".55"/>
</DoubleAnimationUsingKeyFrames>
</Storyboard>
</vsm:VisualState>
</vsm:VisualStateGroup>
<vsm:VisualStateGroup x:Name="SelectionStates">
<vsm:VisualState x:Name="Unselected"/>
<vsm:VisualState x:Name="Selected">
<Storyboard>
<DoubleAnimationUsingKeyFrames Storyboard.TargetName="fillColor2" Storyboard.TargetProperty="Opacity">
<SplineDoubleKeyFrame KeyTime="0" Value=".75"/>
</DoubleAnimationUsingKeyFrames>
</Storyboard>
</vsm:VisualState>
</vsm:VisualStateGroup>
<vsm:VisualStateGroup x:Name="FocusStates">
<vsm:VisualState x:Name="Focused">
<Storyboard>
<ObjectAnimationUsingKeyFrames Duration="0" Storyboard.TargetName="FocusVisualElement" Storyboard.TargetProperty="Visibility">
<DiscreteObjectKeyFrame KeyTime="0">
<DiscreteObjectKeyFrame.Value>
<Visibility>Visible</Visibility>
</DiscreteObjectKeyFrame.Value>
</DiscreteObjectKeyFrame>
</ObjectAnimationUsingKeyFrames>
</Storyboard>
</vsm:VisualState>
<vsm:VisualState x:Name="Unfocused"/>
</vsm:VisualStateGroup>
</vsm:VisualStateManager.VisualStateGroups>
<Rectangle x:Name="fillColor" IsHitTestVisible="False" Opacity="0" RadiusX="1" RadiusY="1" Fill="Transparent"/>
<Rectangle x:Name="fillColor2" IsHitTestVisible="False" Opacity="0" Fill="#FF00FF00" RadiusX="1" RadiusY="1"/>
<ContentPresenter HorizontalAlignment="Left" Margin="{TemplateBinding Padding}" x:Name="contentPresenter" Content="{TemplateBinding Content}" ContentTemplate="{TemplateBinding ContentTemplate}"/>
<Rectangle x:Name="FocusVisualElement" Visibility="Collapsed" Stroke="#FF6DBDD1" StrokeThickness="1" RadiusX="1" RadiusY="1"/>
</Grid>
</ControlTemplate>
</Setter.Value>
</Setter>
</Style>
</UserControl.Resources>
<Grid x:Name="LayoutRoot" Background="White">
<ListBox x:Name="ListBoxTest">
<ListBoxItem Content="Some String" Style="{StaticResource ListBoxItemStyleTransparent}" />
</ListBox>
</Grid>
</UserControl>
The fillColor Rectangle specifies the color to use when the user mouses-over a ListBoxItem. In the code above I have set this to be Transparent so no color will appear when you mouse over the ListBoxItem.
The fillColor2 specifies the color to use when a ListBoxItem is selected. In the code above I've specified #FF00FF00 so the color will be bright green when a ListBoxItem is selected.
In your situation you would set the Fill property of the fillColor2 Rectangle to be Transparent to simulate no color when the user selects an item.