Git gets file hungry every once in a while...

Question

I have a recurring problem with my Git repositories. I develop in Windows and my production site is under Linux. Several times it has happened that git was showing all files tracked as modified. I thought this was because of a conf issue or conflict between Windows and Linux, but then this morning, when I checked the Linux repo, it was showing all files as modified.

To add insult to injury, the two Linux repos I use (1 for prod, 1 for test) were showing the same. I had no other choice but to commit all the files, as a hard reset or a checkout were making no changes to the working directory (yup, I pretty much sucks at this). This is the result of the commit:

Created commit #######: Git, you are so mean...
1521 files changed, 302856 insertions(+), 302856 deletions(-)

Any ideas on how to sort this out next time it happens?

Answer 1

Sounds like a line-ending issue. Check man git-config for core.autocrlf.

Answer 2

As Bombe says, this sounds like a line-ending problem. The simplest discussion of this that I've seen is this guide at Github.

You want to set core.autocrlf on your Windows system so that Git will automatically change line endings to CRLF when you checkout from the repo into the working directory, and conversely change all CRLFs to LF when you commit files to the repo:

$ git config --global core.autocrlf true

Then you can ensure that your repo has consistent line endings either by cloning from your Linux repo or by git reset, which checks out a fresh working copy which applies the new autocrlf setting to all the LFs:

$ git reset --hard HEAD

EDIT -- If Git doesn't recognize a file as binary, autocrlf can corrupt the file by changing what it thinks are CRLF or LFs. If your repo includes unusual binary formatted file types, declare their conversion type explicitly in .gitattributes, as described in the manpage.

Answer 3

I think the real issue that you need to address, is how are the files different and is the difference what you are expecting to see?

The traditional default is that git does not alter file contents at all on a git add to the repository. More recent windows git installers enable core.autocrlf which translates unix to windows line endings on a checkout, and the reverse on addition to the repository.

For this reason, if you have more untracked changes that you expect, it is often a good idea to git add all the pending files (e.g. via a git add -u).

At this stage any clean/smudge filters will have be applied and git diff --cached should give a reasonable diff.

If you have staged files that git thinks are different, but the difference is not visible, you may want to have a look at the raw bytes to see if there are any differences in invisible characters.

You might use a tool such as hexdump for this.

Suppose that myfile.txt has differences which are not visible, you might want to try something like this.

# Extract raw versions of the differing files and hexdump to some temporary files
git cat-file blob :myfile.txt | hexdump -C >myfile-stagetmp.bytes
git cat-file blob HEAD:myfile.txt | hexdump -C >myfile-headtmp.bytes

# Diff them. (Yes, you don't have to use git diff!)
git diff --no-index myfile-stagetmp.bytes myfile-headtmp.bytes