SQL temp table sharing accross different SQL readers

Question

I am trying to do a many different queries on a result set which has a very large creation time. To get performance gains I wish to use a temp table and just do many queries on this temp table.

Seems pretty standard. Yet I am struggling to share this temp table in dynamic sql. As I understand it, each SqlCommand object executes in its own thread and so the temp table is in a different scope - thus making it inaccessible from the query thread.

I tried using a global temporary table and that works great, but not ideal?

How can I share a local temporary table between dynamic SQL queries?

My intent:

using (var conn = new SqlClient.SqlConnection("..."))
{
    // Creation involes many table joins in reality
    String creationScript = "SELECT * FROM FooTable INTO #MyTemp";
    SqlCommand createTempTbl = new SqlCommand(creationScript, conn);
    createTempTbl.ExecuteNonQuery();

    String query1 = "SELECT * FROM #MyTemp where id=@id";
    SqlCommand query1Comm = new SqlCommand(query1, conn);
    query1Comm.Parameters.Add("@id", ...);

    String query2 = "SELECT * FROM #MyTemp where name=@name";
    SqlCommand query2Comm = new SqlCommand(query2, conn);
    query2Comm.Parameters.Add("@name", ...);

    // And so on the queries go

} // Now want #MyTemp to be destroyed

Answer 1

You could try using a global temporary table (ie, use ##MyTemp rather than #MyTemp in your queries), with this caveat:

Global temporary tables are automatically dropped when the session that created the table ends and all other tasks have stopped referencing them. The association between a task and a table is maintained only for the life of a single Transact-SQL statement. This means that a global temporary table is dropped at the completion of the last Transact-SQL statement that was actively referencing the table when the creating session ended.

---

EDIT: Oops, missed the fact that you've already tried global temp tables.

How about moving all of your logic into a single stored procedure which creates/populates the temp table and then runs the queries and returns multiple resultsets to the client code?

Answer 2

What is missing from your question is the lifecycle of the created table. If you will have it sticking around for a while, then it is not quite a temp table, it is a work table that you populate and use. I would not use a temp table at all, just a regular table that gets created by the SELECT INTO and used by everyone else until it gets dropped (if ever).

Answer 3

An alternative that I've used successfully is to create a work table in TempDb, and use it as if it is a global temp table (e.g., "TempDb.dbo.MyTable"). Remember that user tables are dropped when SQL Server reboots.

Answer 4

I know it's a while since this one was posted but the answer, I believe, is quite simple.

I surmise you are using the MS Enterprise Library to access the database, this explains why the temp table doesn’t exist between commands. The Enterprise Library EXPLICITLY closes the connection to the DB (puts it back in the pool) when the command finishes. That is, UNLESS you put the commands into a transaction. If you use ADO.NET directly (by opening the connection, building and executing the commands, then closing the connection) you do not get this problem (it’s up to you when the connection closes – which is more risky). Here is some code written using the MS Enterprise Library and a transaction (sorry, VB.NET):

// Get a reference to the database
Dim sqlNET As New Sql.SqlDatabase("*Your Connection String Here...*")

//Specify the transaction options
Dim oTranOpt As TransactionOptions = New TransactionOptions
//What type of isolation the transaction should have (V. Important):
oTranOpt.IsolationLevel = IsolationLevel.ReadUncommitted //This one doesn't place locks on DB but allows dirty reads
//How long the transaction has to complete before implicitly failing (h,m,s):
oTranOpt.Timeout = New TimeSpan(0, 0, 30)

//Start the scope of the transation
Using oTranScope As TransactionScope = New TransactionScope(TransactionScopeOption.Required, oTranOpt)

    //Create the connection to the DB. Not abs. necessary. EL will create one but best to do so.
    Using Conn As Common.DbConnection = sqlNET.CreateConnection

        //Create a Temp Table
        sqlNET.ExecuteNonQuery(CommandType.Text, "SELECT * INTO #MyTemp FROM FooTable")

        //Get results from table, e.g.
        Dim intCount As Integer = sqlNET.ExecuteScalar(CommandType.Text, "Select Count(*) from #MyTemp")

        MsgBox(intCount)

        //Flag transaction as successful (causes a commit when scope is disposed)
        oTranScope.Complete()

    End Using //Disposes the connection

End Using //If this point is reached without hitting the oTranScope.Complete - the transaction is rolled back and locks, if any, are released.

If you were to take out the transaction scope, the code would fail on the Select count(*) as the table no longer exists. Specifying the scope keeps the connection open between command calls.

I hope this helps someone.

Neil.