git rebase, "would be overwritten", and "No changes - did you forget to use 'git add'?"

Question

git rebase does not appear to work properly in certain cases where a file is added to the repository, then removed from the repository, then added to the working directory (but not the repository).

Here's a more specific description of my problem:

• if a branch is created and switched to from some trunk,

• and a file X is added and committed in the branch,

• and subsequently X is removed and committed in the branch,

• and X is again created in the working directory, but not added or committed,

• and the trunk branch advances,

• then

• a rebase performed using the advanced trunk as the base will fail because it will refuse to overwrite X,

• and the rebase cannot be continued even if the working directory X is removed or moved out of the way.

Here's a script to reproduce my problem on the command line:

git init
echo foo > foo.txt
git add .
git commit -m 'foo'
echo foo >> foo.txt
git add .
git commit -m 'foo foo'
git checkout -b topic HEAD^
git log
echo bar > bar.txt
echo baz > baz.txt
git add .
git commit -m 'bar baz'
git rm bar.txt
git commit -m '-bar' 
echo bar > bar.txt
git rebase master 
# the following output is emitted:
# First, rewinding head to replay your work on top of it...
# Applying: bar baz
# Using index info to reconstruct a base tree...
# Falling back to patching base and 3-way merge...
# error: Untracked working tree file 'bar.txt' would be overwritten by merge.  Aborting
# Failed to merge in the changes.
# Patch failed at 0001 bar baz
# 
# When you have resolved this problem run "git rebase --continue".
rm bar.txt
git rebase --continue
# the following output is emitted:
# Applying: bar baz
# No changes - did you forget to use 'git add'?
# 
# When you have resolved this problem run "git rebase --continue".
# If you would prefer to skip this patch, instead run "git rebase --skip".
# To restore the original branch and stop rebasing run "git rebase --abort".

I know I can abort the rebase using git rebase --abort, remove bar.txt, and then git rebase master again. But how can I continue the rebase without aborting it first?

Answer 1

You need to commit before attempting to rebase. It will work sometimes if you don't, but you shouldn't do that.

If you're uncomfortable committing for some reason, you could at least use stash (which does roughly the same thing).

Answer 2

In order for GIT to proceed with rebase after file removal I added a whitespace change and then "git add" the file with that change.

Answer 3

git rebase is working correctly. It is protecting your untracked file from being destroyed while visiting a commit that wants to write out a file with that same pathname.

There does not seem to be a way to retry just the most recent step of the rebase. Usually when a rebase pauses, it will leave conflicts in the index. But in this case, it can not flag this problem in the index because doing so would convert your untracked file into a tracked file. This is probably a bit of a rough spot in the UI of git rebase. You could dig into the directory it uses to store its internal state (.git/rebase-apply) and manually apply the “current” patch, but aborting and restart is probably easier (but maybe not faster if you are in the middle of rebasing a very long series).

---

If the addition of bar.txt is considered a mistake, then you might consider using git rebase -i to tease apart and drop the addition and removal of bar.txt since you are rewriting history anyway.
You will still run into the conflict, but the methods below can also be applied with git rebase -i. The script at the end would have to be split into two parts (“setup temp/” and “incorporate rebase result” since an interactive rebase will usually require multiple commands between those two sections.

---

Move the conflicting file aside temporarily and redo the rebase.

mv bar.txt +bar.txt
git rebase --abort
git rebase master

If you expect many such conflicting files then you might consider doing your rebase in a separate clone where you can be certain that you will not have any untracked files. Perhaps the trickiest part is checking that your untracked files do not conflict with the result of the rebase (the git checkout rebased-topic accomplishes this; it aborts if the untracked files conflict with the rebase result).

: "Note: will destroy
:     * 'rebased-topic' branch
:     * 'temp' directory"
( set -x &&
  : '*** Clearing temp/' &&
  rm -rf temp/ &&

  : '*** Making sure we have topic checked out' &&
  git checkout topic

  : '*** Cloning into temp/' &&
  git clone . temp && 

  : '*** Rebasing topic onto master in temp/ clone' &&
  ( cd temp &&
  git rebase origin/master
  ) &&

  : '*** Fetching rebase result from topic/ into local rebased-topic branch' &&
  git fetch -f temp topic:rebased-topic &&

  : '*** Checking rebased-topic for conflicts with untracked files' &&
  git checkout rebased-topic &&

  : '*** Resetting topic to tip of rebased-topic' &&
  git branch -f topic rebased-topic &&

  : '*** Returning to topic branch' &&
  git checkout topic &&

  : '*** Deleting rebased-topic branch' &&
  git branch -d rebased-topic &&

  : '*** Deleting temp/' &&
  rm -rf temp/
)

Answer 4

I found a solution: apply the troubled commit's patch manually and add it to the index.

$ patch -p1 < .git/rebase-apply/patch 
patching file bar.txt
patching file baz.txt
$ git add bar.txt baz.txt
$ git rebase --continue
Applying: bar baz
Applying: -bar