Okay, I got as far as the saving to local folder and deleting from message. I haven't worked out buttons yet, but I'm sure it's not the hardest thing in the world...
So I would check out the VBA documentation on Attachment Methods, specifically the one on SaveAsFile, as it has a full example that I used to test things out. The two methods available are the exact ones you need:
SaveAsFile
and
Delete
But since VBA makes nothing simple, using those two lines requires 15 others. 
Also there is a REALLY great site called outlookcode.com. The site admin is a VBA/Outlook wizard and she will personally answer your questions if they sit on the forums for a more than a day (not a guarantee, just my experience). The site is full of sources and other people's code, etc.
Here is what I wrote to try out what you had in mind, based on the sample from MSDN which I added the delete method, making it a one click save/delete:
Sub getAttatchment()
    Dim myInspector As Outlook.Inspector
    Dim myItem As Outlook.MailItem
    Dim myAttachments As Outlook.Attachments
    Set myInspector = Application.ActiveInspector
    If Not TypeName(myInspector) = "Nothing" Then
        If TypeName(myInspector.CurrentItem) = "MailItem" Then
            Set myItem = myInspector.CurrentItem
            Set myAttachments = myItem.Attachments
                myAttachments.Item(1).SaveAsFile Environ("HOMEPATH") _ 
                & "\My Documents\" & myAttachments.Item(1).DisplayName
                myAttachments.Item(1).Delete
        Else
            MsgBox "The item is of the wrong type."
        End If
    End If
End Sub
Be aware that the original sample has a dialog box to ask the user if they are sure they want to save as it will overwrite any files with the same name. I deleted it to simplify the code a bit.