Sharepoint web services -- The HTTP request is unauthorized with client authentication scheme 'Ntlm'. The authentication header received from the server was 'NTLM'.

Question

I know there's a lot of questions on SO similar to this, but I couldn't find one for this particular issue.

A couple of points, first:

• I have no control over our Sharepoint server. I cannot tweak any IIS settings.
• I believe our IIS server version is IIS 7.0.
• Our Sharepoint Server is anticipating requests via NTLM.
• Our Sharepoint Server is on the same domain as my client computer.
• I am using .NET Framework 3.5, Visual Studio 2008

I am trying to write a simple console app to manipulate Sharepoint data using Sharepoint Web Services. I have added the Service Reference, and the following is my app.config:

<system.serviceModel>
    <bindings>
        <basicHttpBinding>
            <binding name="ListsSoap" closeTimeout="00:01:00" openTimeout="00:01:00"
                receiveTimeout="00:10:00" sendTimeout="00:01:00" allowCookies="false"
                bypassProxyOnLocal="false" hostNameComparisonMode="StrongWildcard"
                maxBufferSize="65536" maxBufferPoolSize="524288" maxReceivedMessageSize="65536"
                messageEncoding="Text" textEncoding="utf-8" transferMode="Buffered"
                useDefaultWebProxy="true">
                <readerQuotas maxDepth="32" maxStringContentLength="8192" maxArrayLength="16384"
                    maxBytesPerRead="4096" maxNameTableCharCount="16384" />
                <security mode="Transport">
                    <transport clientCredentialType="Ntlm" proxyCredentialType="Ntlm" />
                </security>
            </binding>
        </basicHttpBinding>
    </bindings>
    <client>
        <endpoint address="https://subdomain.companysite.com/subsite/_vti_bin/Lists.asmx"
            binding="basicHttpBinding" bindingConfiguration="ListsSoap"
            contract="ServiceReference1.ListsSoap" name="ListsSoap" />
    </client>
</system.serviceModel>

This is my code:

static void Main(string[] args)
{
    using (var client = new ListsSoapClient())
    {
        client.ClientCredentials.Windows.ClientCredential = new NetworkCredential("username", "password", "domain");
        client.GetListCollection();
    }
}

When I call GetListCollection(), the following MessageSecurityException gets thrown:

The HTTP request is unauthorized with client authentication scheme 'Ntlm'.
The authentication header received from the server was 'NTLM'.

With an inner WebException:

"The remote server returned an error: (401) Unauthorized."

I've tried various bindings and various code tweaks to try to authenticate properly, but to no avail. I'll list those below.

---

I've tried the following steps:

Using a native Win32 Impersonator before creating the client

using (new Impersonator.Impersonator("username", "password", "domain"))
using (var client = new ListsSoapClient())
{
    client.ClientCredentials.Windows.ClientCredential = new NetworkCredential("dpincas", "password", "domain");
    client.GetListCollection();
}

This produced the same error message.

---

Setting TokenImpersonationLevel for my client credentials

using (var client = new ListsSoapClient())
{
    client.ClientCredentials.Windows.AllowedImpersonationLevel = TokenImpersonationLevel.Impersonation;
    client.GetListCollection();
}

This produced the same error message.

---

Using security mode=TransportCredentialOnly

<security mode="TransportCredentialOnly">
    <transport clientCredentialType="Ntlm" />
</security>

This resulted in a different error message:

The provided URI scheme 'https' is invalid; expected 'http'.
Parameter name: via

However, I need to use https, so I cannot change my URI scheme.

---

I've tried some other combinations that I can't remember, but I'll post them when I do. I'm really at wits end here. I see a lot of links on Google that say "switch to Kerberos", but my server seems to only be accepting NTLM, not "Negotiate" (as it would say if it was looking for Kerberos), so that is unfortunately not an option.

Any help out there, folks?

Answer 1

Visual Studio 2005

1. Create a new console application project in Visual Studio
2. Add a "Web Reference" to the Lists.asmx web service.
   • Your URL will probably look like: http://servername/sites/SiteCollection/SubSite/_vti_bin/Lists.asmx
   • I named my web reference: ListsWebService
3. Write the code in program.cs (I have an Issues list here)

Here is the code.

using System;
using System.Collections.Generic;
using System.Text;
using System.Xml;

namespace WebServicesConsoleApp
{
    class Program
    {
        static void Main(string[] args)
        {
            try
            {
                ListsWebService.Lists listsWebSvc = new WebServicesConsoleApp.ListsWebService.Lists();
                listsWebSvc.Credentials = System.Net.CredentialCache.DefaultNetworkCredentials;
                listsWebSvc.Url = "http://servername/sites/SiteCollection/SubSite/_vti_bin/Lists.asmx";
                XmlNode node = listsWebSvc.GetList("Issues");
            }
            catch (Exception ex)
            {
                Console.WriteLine(ex.ToString());
            }
        }
    }
}

---

Visual Studio 2008

1. Create a new console application project in Visual Studio
2. Right click on References and Add Service Reference
3. Put in the URL to the Lists.asmx service on your server
   • Ex: http://servername/sites/SiteCollection/SubSite/_vti_bin/Lists.asmx
4. Click Go
5. Click OK
6. Make the following code changes:

Change your app.config file from:

<security mode="None">
    <transport clientCredentialType="None" proxyCredentialType="None"
        realm="" />
    <message clientCredentialType="UserName" algorithmSuite="Default" />
</security>

To:

<security mode="TransportCredentialOnly">
  <transport clientCredentialType="Ntlm"/>
</security>

Change your program.cs file and add the following code to your Main function:

ListsSoapClient client = new ListsSoapClient();
client.ClientCredentials.Windows.ClientCredential = System.Net.CredentialCache.DefaultNetworkCredentials;
client.ClientCredentials.Windows.AllowedImpersonationLevel = System.Security.Principal.TokenImpersonationLevel.Impersonation;
XmlElement listCollection = client.GetListCollection();

Add the using statements:

using [your app name].ServiceReference1;
using System.Xml;

Reference: http://sharepointmagazine.net/technical/development/writing-caml-queries-for-retrieving-list-items-from-a-sharepoint-list

Answer 2

If I recall correctly, there are some issues with adding SharePoint web services as a VS2K8 "Service Reference". You need to add it as an old-style "Web Reference" to work properly.

Answer 3

I have the same setup that you do, and this works fine for me. I think that maybe the problem lies somewhere on your moss configuration or on your network.

You said that moss resides on the same domain as your application. If you have access to the site with your user (that is logged into your machine)... have you tried:

client.ClientCredentials.Windows.ClientCredential = System.Net.CredentialCache.DefaultNetworkCredentials;

Answer 4

Try this

<client>
  <endpoint>
    <identity>
      <servicePrincipalName value="" />
    </identity>
  </endpoint>
</client>

I've encountered this error before when working in a webfarm and this fixed it for me.

Answer 5

I had exactly the same issue last week - http://stackoverflow.com/questions/3197484/wcf-program-behaves-strangely-on-one-server-why

For me the solution was rather simple. Sharepoint has its own set of permissions. My client tried to log on as a user that wasn't explicitly given access to the webservice through Sharepoint's administration panel.

I added the user to Sharepoint's whitelist and bang - it just worked.

Even if that isn't the issue, please note that

The HTTP request is unauthorized with client authentication scheme ‘Ntlm’. The authentication header received from the server was ‘NTLM’.

Means (in English) that you simply don't have permission. Your protocol is probably right - your user just doesn't have permissions.

Answer 6

I would try to connect to your Sharepoint site with this tool here. If that works you can be sure that the problem is in your code / configuration. That maybe does not solve your problem immediately but it rules out that there is something wrong with the server. Assuming that it does not work I would investigate the following:

• Does your user really have enough rights on the site?
• Is there a proxy that interferes? (Your configuration looks a bit like there is a proxy. Can you bypass it?)

I think there is nothing wrong with using security mode Transport, but I am not so sure about the proxyCredentialType="Ntlm", maybe this should be set to None.

Answer 7

I have had this issue before.

client.ClientCredentials.Windows.AllowedImpersonationLevel = TokenImpersonationLevel.Impersonation;

do this against your wcf proxy before making the call.

Answer 8

After a lot of trial and error, followed by a stagnant period while I waited for an opportunity to speak with our server guys, I finally had a chance to discuss the problem with them and asked them if they wouldn't mind switching our Sharepoint authentication over to Kerberos.

To my surprise, they said this wouldn't be a problem and was in fact easy to do. They enabled Kerberos and I modified my app.config as follows:

<security mode="Transport">
    <transport clientCredentialType="Windows" />
</security>

For reference, my full serviceModel entry in my app.config looks like this:

<system.serviceModel>
    <bindings>
        <basicHttpBinding>
            <binding name="TestServerReference" closeTimeout="00:01:00" openTimeout="00:01:00"
             receiveTimeout="00:10:00" sendTimeout="00:01:00" allowCookies="false"
             bypassProxyOnLocal="false" hostNameComparisonMode="StrongWildcard"
             maxBufferSize="2000000" maxBufferPoolSize="2000000" maxReceivedMessageSize="2000000"
             messageEncoding="Text" textEncoding="utf-8" transferMode="Buffered"
             useDefaultWebProxy="true">
                <readerQuotas maxDepth="32" maxStringContentLength="8192" maxArrayLength="16384"
                 maxBytesPerRead="4096" maxNameTableCharCount="16384" />
                <security mode="Transport">
                    <transport clientCredentialType="Windows" />
                </security>
            </binding>
        </basicHttpBinding>
    </bindings>
    <client>
        <endpoint address="https://path/to/site/_vti_bin/Lists.asmx"
         binding="basicHttpBinding" bindingConfiguration="TestServerReference"
         contract="TestServerReference.ListsSoap" name="TestServerReference" />
    </client>
</system.serviceModel>

After this, everything worked like a charm. I can now (finally!) utilize Sharepoint Web Services. So, if anyone else out there can't get their Sharepoint Web Services to work with NTLM, see if you can convince the sysadmins to switch over to Kerberos.