How do I pull from a Git repository through an HTTP proxy?

Question

Note: while the use-case described is about using submodules within a project, the same applies to a normal git clone of a repository over HTTP.

I have a project under Git control. I'd like to add a submodule:

git submodule add http://github.com/jscruggs/metric_fu.git vendor/plugins/metric_fu

But I get

...
got 1b0313f016d98e556396c91d08127c59722762d0
got 4c42d44a9221209293e5f3eb7e662a1571b09421
got b0d6414e3ca5c2fb4b95b7712c7edbf7d2becac7
error: Unable to find abc07fcf79aebed56497e3894c6c3c06046f913a under http://github.com/jscruggs/metri...
Cannot obtain needed commit abc07fcf79aebed56497e3894c6c3c06046f913a
while processing commit ee576543b3a0820cc966cc10cc41e6ffb3415658.
fatal: Fetch failed.
Clone of 'http://github.com/jscruggs/metric_fu.git' into submodule path 'vendor/plugins/metric_fu'

I have my HTTP_PROXY set up:

c:\project> echo %HTTP_PROXY%
http://proxy.mycompany:80

I even have a global Git setting for the http proxy:

c:\project> git config --get http.proxy
http://proxy.mycompany:80

Has anybody gotten HTTP fetches to consistently work through a proxy? What's really strange is that a few project on GitHub work fine (awesome_nested_set for example), but others consistently fail (rails for example).

Answer 1

Could your proxy be censoring some words or something, thereby only affecting those repos where those words happen to show up as part of the byte stream?

Answer 2

It looks like you're using a mingw compile of Git on windows (or possibly another one I haven't heard about). There are ways to debug this: I believe all of the http proxy work for git is done by curl. Set this environment variable before running git:

GIT_CURL_VERBOSE=1

This should at least give you an idea of what is going on behind the scenes.

Answer 3

I get a whole slew of 200s:

< HTTP/1.1 200 OK
< Server: nginx/0.6.31
< Date: Wed, 24 Sep 2008 16:49:17 GMT
< Content-Type: text/plain; charset=utf-8
< Last-Modified: Sat, 20 Sep 2008 00:32:44 GMT
< Cache-Control: no-cache
< Accept-Ranges: bytes
< Content-Length: 134
< Proxy-Connection: Keep-Alive
< Connection: Keep-Alive
< Set-Cookie: _github_sess=BAh7BiIKZmxhc2hJQzonQWN0aW9uQ29udHJvbGxlcjo6Rmxhc2g6OkZsYXNo%250ASGFzaHsABjoKQHVzZWR7AA%253D%253D--4d8802a39cf2ae
2680fcdf915bd7f8472555cf65; domain=.github.com; path=/; expires=Wed, 01 Jan 2020 08:00:00 GMT
<
< HTTP/1.1 200 OK
< Server: nginx/0.6.31
< Date: Wed, 24 Sep 2008 16:49:17 GMT
< Content-Type: text/plain; charset=utf-8
< Last-Modified: Sat, 20 Sep 2008 00:32:44 GMT
< Cache-Control: no-cache
< Accept-Ranges: bytes
< Content-Length: 346
< Proxy-Connection: Keep-Alive
< Connection: Keep-Alive
< Set-Cookie: _github_sess=BAh7BiIKZmxhc2hJQzonQWN0aW9uQ29udHJvbGxlcjo6Rmxhc2g6OkZsYXNo%250ASGFzaHsABjoKQHVzZWR7AA%253D%253D--4d8802a39cf2ae
2680fcdf915bd7f8472555cf65; domain=.github.com; path=/; expires=Wed, 01 Jan 2020 08:00:00 GMT

Followed by some stuff that makes less sense, other than it doesn't work:

* Connection #4 to host proxy.mycompany left intact
* Connection #5 to host proxy.mycompany left intact
* Connection #8 to host proxy.mycompany left intact
* Connection #7 to host proxy.mycompany left intact
* Couldn't find host github.com in the _netrc file, using defaults
* Closing connection #9
* Connection (#9) was killed to make room (holds 20)
* About to connect() to github.com port 80 (#9)
*   Trying 129.83.31.19... got 1b0313f016d98e556396c91d08127c59722762d0
got 4c42d44a9221209293e5f3eb7e662a1571b09421
got 1f185940888f5d023b2d00f52af125434f22d9e4
got b0d6414e3ca5c2fb4b95b7712c7edbf7d2becac7
* Connected to github.com (129.83.31.19) port 80 (#9)
* Couldn't find host github.com in the _netrc file, using defaults
* Closing connection #10
* Connection (#10) was killed to make room (holds 20)
* About to connect() to github.com port 80 (#10)
*   Trying 129.83.31.19... > GET http://github.com/jscruggs/metric_fu.git/objects/ab/c07fcf79aebed56497e3894c6c3c06046f913a HTTP/1.1
User-Agent: git/1.5.6.1.1071.g76fb
Host: github.com
Accept: */*
Proxy-Connection: Keep-Alive

* Connected to github.com (129.83.31.19) port 80 (#10)
* Couldn't find host github.com in the _netrc file, using defaults
* Closing connection #11
* Connection (#11) was killed to make room (holds 20)
* About to connect() to github.com port 80 (#11)
*   Trying 129.83.31.19... * Connected to github.com (129.83.31.19) port 80 (#11)
* Couldn't find host github.com in the _netrc file, using defaults
* Closing connection #12
* Connection (#12) was killed to make room (holds 20)
* About to connect() to github.com port 80 (#12)
*   Trying 129.83.31.19... > GET http://github.com/jscruggs/metric_fu.git/objects/33/7ac75414fd8283727eefcabcbd4c7d7b9998af HTTP/1.1
User-Agent: git/1.5.6.1.1071.g76fb
Host: github.com
Accept: */*
Proxy-Connection: Keep-Alive

* Connected to github.com (129.83.31.19) port 80 (#12)
* Couldn't find host github.com in the _netrc file, using defaults
* Closing connection #13
* Connection (#13) was killed to make room (holds 20)
* About to connect() to github.com port 80 (#13)
*   Trying 129.83.31.19... > GET http://github.com/jscruggs/metric_fu.git/objects/6d/9af6049be159dacba051f88c705e91f85b6fd6 HTTP/1.1
User-Agent: git/1.5.6.1.1071.g76fb
Host: github.com
Accept: */*
Proxy-Connection: Keep-Alive

* Connected to github.com (129.83.31.19) port 80 (#13)
> GET http://github.com/jscruggs/metric_fu.git/objects/46/0ade6e3ac63e1989dc48a7a5c660b354e65250 HTTP/1.1
User-Agent: git/1.5.6.1.1071.g76fb
Host: github.com
Accept: */*
Proxy-Connection: Keep-Alive

> GET http://github.com/jscruggs/metric_fu.git/objects/d9/338a0886df9e4abecbbdda426a8ee371ae0796 HTTP/1.1
User-Agent: git/1.5.6.1.1071.g76fb
Host: github.com
Accept: */*
Proxy-Connection: Keep-Alive

* The requested URL returned error: 404
* Closing connection #9
error: Unable to find abc07fcf79aebed56497e3894c6c3c06046f913a under http://github.com/jscruggs/metric_fu.git
Cannot obtain needed commit abc07fcf79aebed56497e3894c6c3c06046f913a
while processing commit ee576543b3a0820cc966cc10cc41e6ffb3415658.
fatal: Fetch failed.

Answer 4

This isn't a problem with your proxy. It's a problem with github (or git). It fails for me on git-1.6.0.1 on linux as well. Bug is already reported (by you no less).

Make sure to delete your pasties, they're already on google. Edit: Must've been dreaming, i guess you can't delete them. Use Gist instead?

Answer 5

What finally worked was setting the $http_proxy environment variable. I had set $HTTP_PROXY correctly, but git apparently likes the lower-case version better.

Answer 6

You can also set the HTTP proxy that Git uses in global configuration property http.proxy:

git config --global http.proxy $http_proxy