You can delete files during uninstall using a custom action. One of the easiest ways to set this up is with an Installer Class.
This article by Arnaldo Sandoval is a little out dated, but it is still a pretty good overview on how to use installer classes to implement custom actions. It even includes a section about cleaning up files on uninstall.
However, instead of overriding methods in the Installer class, it is better to add event listeners. Where you get the "saved state" is also a little different. For example, where the article describes overriding the Install method to capture the TargetDir, instead of:
public override void Install(System.Collections.IDictionary stateSaver)
{
base.Install(stateSaver);
stateSaver.Add("TargetDir", Context.Parameters["DP_TargetDir"].ToString());
}
You would create a method similar to:
private void onBeforeInstall(object sender, InstallEventArgs args)
{
// The saved state dictionary is a property on the event args
args.SavedState.Add("TargetDir", Context.Parameters["DP_TargetDir"].ToString());
}
And register it in the constructor:
public InstallerClass() : base()
{
this.BeforeInstall += new InstallEventHandler(onBeforeInstall);
}
You could also register the events via the Visual Studio Property editor, if that's more your thing.
The rest of the article is excellent, particularly the sections that discuss all the undocumented "features" of the various install events.