Accessing HTTPS site through Proxy Server

Question

I am adding code to use a proxy server to access the Internet. The code works fine when requesting a file from a normal (HTTP) location, but does not work when accessing a secure location (HTTPS).

This is the code that works just fine:

URL = "http://UnSecureSite.net/file.xml"
Dim wr As HttpWebRequest = CType(WebRequest.Create(URL), HttpWebRequest)
Dim proxy As System.Net.IWebProxy
proxy = WebRequest.GetSystemWebProxy
wr.Proxy.Credentials = System.Net.CredentialCache.DefaultCredentials
Dim ws As HttpWebResponse = CType(wr.GetResponse(), HttpWebResponse)

// (more work here)

As soon as I change the URL to go to HTTPS, I get a 407 returned to me.

Anyone have any ideas?

URL = "https://SecureSite.net/file.xml"
Dim wr As HttpWebRequest = CType(WebRequest.Create(URL), HttpWebRequest)
Dim proxy As System.Net.IWebProxy
proxy = WebRequest.GetSystemWebProxy
wr.Proxy.Credentials = System.Net.CredentialCache.DefaultCredentials
Dim myCache As New CredentialCache()
myCache.Add(New Uri("https://SecureSite.net"), "Basic", New NetworkCredential(UserName, Password))
wr.Credentials = myCache
Dim ws As HttpWebResponse = CType(wr.GetResponse(), HttpWebResponse)

// (more work here)

Answer 1

A HTTPS request through a web-proxy is different from a standard HTTP request. A regular HTTP request will use the GET method. However, a HTTPS request needs to use a CONNECT method. Then, the proxy will merely establish a tunnel to the server. Subsequent messages will be sent directly between the client and the server through the proxy tunnel. The proxy has no way of interpreting the data flowing in between.

Under normal situations:

Client -+- [CONNECT] ---> Proxy --- [DIRECT TCP] -+-> Server
        |                   |                     |
        +-------------[ENCRYPTED TCP]-------------+

I am not familiar enough with the VB code to know if that is what is happening. However, I suspect that it is not. The easiest way to check is to intercept the message being sent to the proxy. Make sure that it begins with a "CONNECT ...".