DotNetOpenAuth RelayParty not working on load balanced cluster

Question

We're trying to move an ASP.NET MVC application, which uses DotNetOpenAuth OpenID Version 3.4.1, from a single server web garden to a physical server cluster held behind a hardware load balancer.

Our old setup (OpenID RP working):

Browser => SHTTP => Server => WebGarden => Nonce/Session Store

Our new setup (OpenID RP not working):

Browser => SHTTP => Load Balancer => HTTP => Cluster Node => WebGarden => Nonce/Session Store DB

When we authenticate with the new setup we are correctly redirected to the OpenID Provider but after authenticated we are redirected back to our cluster (relay party) and get the following exception:

Exception

DotNetOpenAuth.Messaging.ProtocolException: Redirects on POST requests that are to untrusted servers is not supported.
 at DotNetOpenAuth.Messaging.ErrorUtilities.VerifyProtocol(Boolean condition, String message, Object[] args) in c:\TeamCity\buildAgent\work\bf9e2ca68b75a334\src\DotNetOpenAuth\Messaging\ErrorUtilities.cs:line 235
 at DotNetOpenAuth.Messaging.UntrustedWebRequestHandler.GetResponse(HttpWebRequest request, DirectWebRequestOptions options) in c:\TeamCity\buildAgent\work\bf9e2ca68b75a334\src\DotNetOpenAuth\Messaging\UntrustedWebRequestHandler.cs:line 258
 at DotNetOpenAuth.OpenId.ChannelElements.OpenIdChannel.GetDirectResponse(HttpWebRequest webRequest) in c:\TeamCity\buildAgent\work\bf9e2ca68b75a334\src\DotNetOpenAuth\OpenId\ChannelElements\OpenIdChannel.cs:line 277
 at DotNetOpenAuth.Messaging.Channel.RequestCore(IDirectedProtocolMessage request) in c:\TeamCity\buildAgent\work\bf9e2ca68b75a334\src\DotNetOpenAuth\Messaging\Channel.cs:line 542
 at DotNetOpenAuth.Messaging.Channel.Request(IDirectedProtocolMessage requestMessage) in c:\TeamCity\buildAgent\work\bf9e2ca68b75a334\src\DotNetOpenAuth\Messaging\Channel.cs:line 425
 at DotNetOpenAuth.Messaging.Channel.Request[TResponse](IDirectedProtocolMessage requestMessage) in c:\TeamCity\buildAgent\work\bf9e2ca68b75a334\src\DotNetOpenAuth\Messaging\Channel.cs:line 405
 at DotNetOpenAuth.OpenId.ChannelElements.SigningBindingElement.ProcessIncomingMessage(IProtocolMessage message) in c:\TeamCity\buildAgent\work\bf9e2ca68b75a334\src\DotNetOpenAuth\OpenId\ChannelElements\SigningBindingElement.cs:line 154
 at DotNetOpenAuth.Messaging.Channel.ProcessIncomingMessage(IProtocolMessage message) in c:\TeamCity\buildAgent\work\bf9e2ca68b75a334\src\DotNetOpenAuth\Messaging\Channel.cs:line 992
 at DotNetOpenAuth.OpenId.ChannelElements.OpenIdChannel.ProcessIncomingMessage(IProtocolMessage message) in c:\TeamCity\buildAgent\work\bf9e2ca68b75a334\src\DotNetOpenAuth\OpenId\ChannelElements\OpenIdChannel.cs:line 172
 at DotNetOpenAuth.Messaging.Channel.ReadFromRequest(HttpRequestInfo httpRequest) in c:\TeamCity\buildAgent\work\bf9e2ca68b75a334\src\DotNetOpenAuth\Messaging\Channel.cs:line 386
 at DotNetOpenAuth.OpenId.RelyingParty.OpenIdRelyingParty.GetResponse(HttpRequestInfo httpRequestInfo) in c:\TeamCity\buildAgent\work\bf9e2ca68b75a334\src\DotNetOpenAuth\OpenId\RelyingParty\OpenIdRelyingParty.cs:line 501

We have added a machines involved into the trusted machine list and turned off requires ssl but it makes no difference. We even tried removing out nonce store and using a stateless connection, but that didn't work either. We always get the same error.

We suspected the issue is arising as a result of the cluster node having a different IP from the load balancer when it connects to the OpenID Provider, but we're not sure.

Any ideas?

Answer 1

Hi Garth,

The reference to an "untrusted server" is perhaps a little misleading. It doesn't have to do with the whitelist/blacklist in the web.config file, although that was a good guess. In this case everything is an untrusted server because OpenID uses the UntrustedWebRequestHandler, to protect your site against a myriad of attacks that OpenID could otherwise expose your site to.

It looks like your RP is finishing an authentication with an OP by checking the auth response using direct verification (dumb mode), and that the OP endpoint itself is sending a HTTP Redirect response back to your RP. This is not allowed. Does this problem happen for every OP your RP tries to log in with or just this one? Which one(s) exhibit this problem? I should like to talk to the OP owner about what they're doing.

Answer 2

Thanks for the reply, let me give some more info:

We have both the OP's and RP in-house. We have multiple organisations who don't really trust each other, so we distribute the provider to each organisation and then use attribute exchange to pass over the user data (email address, personel number, etc...) without having to have access to each others data stores (usually LDAP) directly.

What is puzzling us is why the setup works fine on a single computer (eg when we connect to a cluster node directly) and not when we connect to the cluster via the hardware load balancer.

We have tried all kinds of different config on both ends, but with no luck so far.