The grip can be removed by setting the attached property ToolBarTray.IsLocked="True"
on the ToolBar.
To remove the Overflow ToggleButton, you will have to remove it in a custom ControlTemplate as sixlettervariables suggests, which if you have blend or can download the Blend 3 Preview is not overly difficult.
You could also just hide the button in the loaded event of the ToolBar, though whichever route you take, you should also set the attached property ToolBar.OverflowMode="Never"
on the ToolBar's menu, so that items cannot accidentally overflow into an unreachable area.
<ToolBarPanel DockPanel.Dock="Top">
<ToolBar ToolBarTray.IsLocked="True" Loaded="ToolBar_Loaded">
<Menu ToolBar.OverflowMode="Never">
<MenuItem Header="File" />
<MenuItem Header="New" />
</Menu>
</ToolBar>
</ToolBarPanel>
And set the Overflow ToggleButton to collapsed:
private void ToolBar_Loaded(object sender, RoutedEventArgs e)
{
ToolBar toolBar = sender as ToolBar;
var overflowGrid = toolBar.Template.FindName("OverflowGrid", toolBar) as FrameworkElement;
if (overflowGrid != null)
{
overflowGrid.Visibility = Visibility.Collapsed;
}
}