Remove a file that I shouldn't have committed to Git

Question

So I did a bad thing.

Somewhere during the course of making a bunch of changes, I realized that a unintended file had snuck into a commit or two. Because I didn't realize this until later, the commits that included the file have now been pushed to the remote. I want & need the commits, I just want to remove this specific file from them.

What I need to do, of course, is to reach into every nook & cranny of my tree (local and remote) and obliterate that file. I've tried a few things using filter-branch and filter-tree, but when I attempt to push the changes get rejected.

What are my options? What am I doing wrong?

Thanks.

UPDATE

At max's request, here's the message I get when attempting to push:

$ git push origin develop
To [email protected]:robwilkerson/cakephp-polyclip-plugin.git
! [rejected]        develop -> develop (non-fast-forward)
error: failed to push some refs to '[email protected]:robwilkerson/cakephp-polyclip-plugin.git'
To prevent you from losing history, non-fast-forward updates were rejected
Merge the remote changes before pushing again.  See the 'Note about fast-forwards' section of 'git push --help' for details.

Answer 1

There is a nice guide over at GitHub, that might help you.

Answer 2

Did you add -f (force) to the git push? Without that flag you can't overwrite "old" commits.

-f, --force
       Usually, the command refuses to update a remote ref that is not an
       ancestor of the local ref used to overwrite it. This flag disables
       the check. This can cause the remote repository to lose commits;
       use it with care.

It can get messy if others already pulled from those repository.

Answer 3

You could you just implement the fix as 'new fix', e.g. remove the file(s) now, and submit the fix to repository.

Then it should get propagated to every branch when they get synced with your version where you made the change.