WCF streaming mode issue

Question

Hello everyone,

I am hosting a WCF service in IIS using basicHttpBinding using streaming mode. The WCF web services queries back end SQL Server 2008 by using ADO.Net and return a DataTable to client side of WCF service.

My understanding of WCF streaming mode are,

1. WCF streaming mode should use only constant number of memory at both client and server side;
2. WCF streaming mode should be able to transfer any number of data between client and server.

Are my understandings correct?

If yes, my following sample break these two rules, I observe two violations,

1. I find my following sample continues to eat memory (from performance tab of task manager, memory usage increases). I run both client and server on the same machine. So, WCF streaming mode does not use constant amount of memory?

2. when transfer 10M rows of a DataTable, there is exception posted below. But transfer 1M rows are fine. So, WCF streaming mode cannot transfer any number of data?

Any ideas what is wrong?

Exception message:

An error occurred while receiving the HTTP response to http://labmachine1/service.svc. This could be due to the service endpoint binding not using the HTTP protocol. This could also be due to an HTTP request context being aborted by the server (possibly due to the service shutting down). See server logs for more details.

{"The underlying connection was closed: An unexpected error occurred on a receive."}

Here is my whole source code for server side, for the server I host in web.config, no change for default values. Since I host in IIS, I am using basicHttpBinding.

public class StudentManagement : IStudentManagement
{
    public DataTable Poll(int Id)
    {
        return MakeParentTable();
    }

    private DataTable MakeParentTable()
    {
        // Create a new DataTable.
        System.Data.DataTable table = new DataTable("ParentTable");
        // Declare variables for DataColumn and DataRow objects.
        DataColumn column;
        DataRow row;

        // Create new DataColumn, set DataType, 
        // ColumnName and add to DataTable.    
        column = new DataColumn();
        column.DataType = System.Type.GetType("System.Int32");
        column.ColumnName = "id";
        column.ReadOnly = true;
        column.Unique = true;
        // Add the Column to the DataColumnCollection.
        table.Columns.Add(column);

        // Create second column.
        column = new DataColumn();
        column.DataType = System.Type.GetType("System.String");
        column.ColumnName = "ParentItem";
        column.AutoIncrement = false;
        column.Caption = "ParentItem";
        column.ReadOnly = false;
        column.Unique = false;
        // Add the column to the table.
        table.Columns.Add(column);

        // Make the ID column the primary key column.
        DataColumn[] PrimaryKeyColumns = new DataColumn[1];
        PrimaryKeyColumns[0] = table.Columns["id"];
        table.PrimaryKey = PrimaryKeyColumns;

        // Create three new DataRow objects and add 
        // them to the DataTable
        for (int i = 0; i <= 10000000; i++)
        {
            row = table.NewRow();
            row["id"] = i;
            row["ParentItem"] = "ParentItem " + i;
            table.Rows.Add(row);
        }

        return table;
    }
}

Client side code:

static void Main(string[] args)
{
    StudentIdentifier identifier = new StudentIdentifier();
    identifier.Id = 100;
    StudentManagementClient client = new StudentManagementClient();

    DataTable student = client.Poll(identifier);

    Console.WriteLine(student.Rows.Count);
}

Streaming mode configuration for client side app.config,

    <basicHttpBinding>
      <binding name="BasicHttpBinding_IStudentManagement" closeTimeout="00:01:00"
        openTimeout="00:20:00" receiveTimeout="01:00:00" sendTimeout="01:00:00"
        allowCookies="false" bypassProxyOnLocal="false" hostNameComparisonMode="StrongWildcard"
        maxBufferSize="1500000000" maxBufferPoolSize="1500000000" maxReceivedMessageSize="1500000000"
        messageEncoding="Mtom" textEncoding="utf-8" transferMode="Streamed"
        useDefaultWebProxy="true">
        <readerQuotas maxDepth="1500000000" maxStringContentLength="1500000000"
          maxArrayLength="1500000000" maxBytesPerRead="1500000000" maxNameTableCharCount="1500000000" />
        <security mode="None">
          <transport clientCredentialType="None" proxyCredentialType="None"
            realm="" />
          <message clientCredentialType="UserName" algorithmSuite="Default" />
        </security>
      </binding>
    </basicHttpBinding>

Server side web.config for streaming mode,

  <basicHttpBinding>
    <binding name="BasicHttpBinding_IStudentManagement" closeTimeout="00:01:00"
        openTimeout="00:20:00" receiveTimeout="01:00:00" sendTimeout="01:00:00"
        allowCookies="false" bypassProxyOnLocal="false" hostNameComparisonMode="StrongWildcard"
        maxBufferSize="1000000000" maxBufferPoolSize="1000000000" maxReceivedMessageSize="1000000000"
        messageEncoding="Mtom" textEncoding="utf-8" transferMode="Streamed"
        useDefaultWebProxy="true">
      <readerQuotas maxDepth="32" maxStringContentLength="1000000000" maxArrayLength="1000000000"
          maxBytesPerRead="4096" maxNameTableCharCount="16384" />
      <security mode="None">
        <transport clientCredentialType="None" proxyCredentialType="None"
            realm="" />
        <message clientCredentialType="UserName" algorithmSuite="Default" />
      </security>
    </binding>
  </basicHttpBinding>

Answer 1

The maxBufferSize attribute is what constrains the amount of memory WCF uses when sending/receiving streamed data. However, you are working loading large objects into memory before sending them to the client.

I would recommend first that you use a DTO (data transfer oject) to send your data to client and that if you need to send that much data to your client that you implement a paging mechanism to transfer the data in managable chunks.

In Theory you can use WCF to send 10 million records to a client, in practice, clients hardly ever need that much data especially all at once.

Answer 2

In stream mode, you have to use system.io.stream as return type or parameter type. obviously, you are not doing so.