views:

1145

answers:

2

I'm currently updating a legacy system which allows users to dictate part of the schema of one of its tables. Users can create and remove columns from the table through this interface. This legacy system is using ADO 2.8, and is using SQL Server 2005 as its database (you don't even WANT to know what database it was using before the attempt to modernize this beast began... but I digress. =) )

In this same editing process, users can define (and change) a list of valid values that can be stored in these user created fields (if the user wants to limit what can be in the field).

When the user changes the list of valid entries for a field, if they remove one of the valid values, they are allowed to choose a new "valid value" to map any rows that have this (now invalid) value in it, so that they now have a valid value again.

In looking through the old code, I noticed that it is extremely vulnerable to putting the system into an invalid state, because the changes mentioned above are not done within a transaction (so if someone else came along halfway through the process mentioned above and made their own changes... well, you can imagine the problems that might cause).

The problem is, I've been trying to get them to update under a single transaction, but whenever the code gets to the part where it changes the schema of that table, all of the other changes (updating values in rows, be it in the table where the schema changed or not... they can be completely unrelated tables even) made up to that point in the transaction appear to be silently dropped. I receive no error message indicating that they were dropped, and when I commit the transaction at the end no error is raised... but when I go to look in the tables that were supposed to be updated in the transaction, only the new columns are there. None of the non-schema changes made are saved.

Looking on the net for answers has, thus far, proved to be a waste of a couple hours... so I turn here for help. Has anyone ever tried to perform a transaction through ADO that both updates the schema of a table and updates rows in tables (be it that same table, or others)? Is it not allowed? Is there any documentation out there that could be helpful in this situation?

EDIT:

Okay, I did a trace, and these commands were sent to the database (explanations in parenthesis)

(I don't know what's happening here, looks like it's creating a temporary stored procedure...?)


declare @p1
int set @p1=180150003 declare @p3 int
set @p3=2 declare @p4 int set @p4=4
declare @p5 int set @p5=-1

(Retreiving the table that holds definition information for the user-generated fields)


exec sp_cursoropen @p1 output,N'SELECT * FROM CustomFieldDefs ORDER BY Sequence',@p3 output,@p4 output,@p5 output select @p1, @p3, @p4, @p5
go

(I think my code was iterating through the list of them here, grabbing the current information)


exec sp_cursorfetch 180150003,32,1,1
go
exec sp_cursorfetch 180150003,32,1,1
go
exec sp_cursorfetch 180150003,32,1,1
go
exec sp_cursorfetch 180150003,32,1,1
go
exec sp_cursorfetch 180150003,32,1,1
go
exec sp_cursorfetch 180150003,32,1,1
go
exec sp_cursorfetch 180150003,32,1,1
go
exec sp_cursorfetch 180150003,32,1,1
go
exec sp_cursorfetch 180150003,1025,1,1
go
exec sp_cursorfetch 180150003,1028,1,1
go
exec sp_cursorfetch 180150003,32,1,1
go

(This appears to be where I'm entering the modified data for the definitions, I go through each and update any changes that occurred in the definitions for the custom fields themselves)


exec sp_cursor 180150003,33,1,N'[CustomFieldDefs]',@Sequence=1,@Description='asdf',@Format='U|',@IsLookUp=1,@Length=50,@Properties='U|',@Required=1,@Title='__asdf',@Type='',@_Version=1
go
exec sp_cursorfetch 180150003,32,1,1
go
exec sp_cursor 180150003,33,1,N'[CustomFieldDefs]',@Sequence=2,@Description='give',@Format='Y',@IsLookUp=0,@Length=0,@Properties='',@Required=0,@Title='_give',@Type='B',@_Version=1
go
exec sp_cursorfetch 180150003,32,1,1
go
exec sp_cursor 180150003,33,1,N'[CustomFieldDefs]',@Sequence=3,@Description='up',@Format='###-##-####',@IsLookUp=0,@Length=0,@Properties='',@Required=0,@Title='_up',@Type='N',@_Version=1
go 
exec sp_cursorfetch 180150003,32,1,1
go
exec sp_cursor 180150003,33,1,N'[CustomFieldDefs]',@Sequence=4,@Description='Testy',@Format='',@IsLookUp=0,@Length=50,@Properties='',@Required=0,@Title='_Testy',@Type='',@_Version=1
go
exec sp_cursorfetch 180150003,32,1,1
go
exec sp_cursor 180150003,33,1,N'[CustomFieldDefs]',@Sequence=5,@Description='you',@Format='U|',@IsLookUp=0,@Length=250,@Properties='U|',@Required=0,@Title='_you',@Type='',@_Version=1
go
exec sp_cursorfetch 180150003,32,1,1
go
exec sp_cursor 180150003,33,1,N'[CustomFieldDefs]',@Sequence=6,@Description='never',@Format='mm/dd/yyyy',@IsLookUp=0,@Length=0,@Properties='',@Required=0,@Title='_never',@Type='D',@_Version=1
go
exec sp_cursorfetch 180150003,32,1,1
go
exec sp_cursor 180150003,33,1,N'[CustomFieldDefs]',@Sequence=7,@Description='gonna',@Format='###-###-####',@IsLookUp=0,@Length=0,@Properties='',@Required=0,@Title='_gonna',@Type='C',@_Version=1
go
exec sp_cursorfetch 180150003,32,1,1
go

(This is where my code removes the deleted through the interface before this saving began]... it is also the ONLY thing as far as I can tell that actually happens during this transaction)


ALTER TABLE CustomizableTable DROP COLUMN _weveknown;

(Now if any of the definitions were altered in such a way that the user-created column's properties need to be changed or indexes on the columns need to be added/removed, it is done here, along with giving a default value to any rows that didn't have a value yet for the given column... note that, as far as I can tell, NONE of this actually happens when the stored procedure finishes.)

go
SELECT * FROM sys.columns WHERE object_id = OBJECT_ID(N'CustomizableTable') AND name = '__asdf'
go
ALTER TABLE CustomizableTable ALTER COLUMN __asdf VarChar(50) NULL
go
IF EXISTS (SELECT * FROM sys.indexes WHERE object_id = OBJECT_ID(N'[dbo].[CustomizableTable]') AND name = N'idx___asdf') CREATE NONCLUSTERED INDEX idx___asdf ON CustomizableTable ( 
__asdf ASC) WITH (PAD_INDEX  = OFF, SORT_IN_TEMPDB = OFF, DROP_EXISTING = OFF, IGNORE_DUP_KEY = OFF, ONLINE = OFF);
go
select * from IF EXISTS (SELECT * FROM sys.indexes WHERE object_id = OBJECT_ID(N'[dbo].[CustomizableTable]') AND name = N'idx___asdf') CREATE NONCLUSTERED INDEX idx___asdf ON 
CustomizableTable ( __asdf ASC) WITH (PAD_INDEX  = OFF, SORT_IN_TEMPDB = OFF, DROP_EXISTING = OFF, IGNORE_DUP_KEY = OFF, ONLINE = OFF);
go
UPDATE CustomizableTable SET [__asdf] = '' WHERE [__asdf] IS NULL
go
SELECT * FROM sys.columns WHERE object_id = OBJECT_ID(N'CustomizableTable') AND name = '_give'
go
ALTER TABLE CustomizableTable ALTER COLUMN _give Bit NULL
go
IF EXISTS (SELECT * FROM sys.indexes WHERE object_id = OBJECT_ID(N'[dbo].[CustomizableTable]') AND name = N'idx__give') DROP INDEX idx__give ON CustomizableTable WITH ( ONLINE = OFF );
go
UPDATE CustomizableTable SET [_give] = 0 WHERE [_give] IS NULL
go
SELECT * FROM sys.columns WHERE object_id = OBJECT_ID(N'CustomizableTable') AND name = '_up'
go
ALTER TABLE CustomizableTable ALTER COLUMN _up Int NULL
go
IF EXISTS (SELECT * FROM sys.indexes WHERE object_id = OBJECT_ID(N'[dbo].[CustomizableTable]') AND name = N'idx__up') DROP INDEX idx__up ON CustomizableTable WITH ( ONLINE = OFF );
go
UPDATE CustomizableTable SET [_up] = 0 WHERE [_up] IS NULL
go
SELECT * FROM sys.columns WHERE object_id = OBJECT_ID(N'CustomizableTable') AND name = '_Testy'
go
ALTER TABLE CustomizableTable ADD _Testy VarChar(50) NULL
go
IF EXISTS (SELECT * FROM sys.indexes WHERE object_id = OBJECT_ID(N'[dbo].[CustomizableTable]') AND name = N'idx__Testy') DROP INDEX idx__Testy ON CustomizableTable WITH ( ONLINE = OFF );
go
UPDATE CustomizableTable SET [_Testy] = '' WHERE [_Testy] IS NULL
go
SELECT * FROM sys.columns WHERE object_id = OBJECT_ID(N'CustomizableTable') AND name = '_you'
go
ALTER TABLE CustomizableTable ALTER COLUMN _you VarChar(250) NULL
go
IF EXISTS (SELECT * FROM sys.indexes WHERE object_id = OBJECT_ID(N'[dbo].[CustomizableTable]') AND name = N'idx__you') DROP INDEX idx__you ON CustomizableTable WITH ( ONLINE = OFF );
go
UPDATE CustomizableTable SET [_you] = '' WHERE [_you] IS NULL
go
SELECT * FROM sys.columns WHERE object_id = OBJECT_ID(N'CustomizableTable') AND name = '_never'
go
ALTER TABLE CustomizableTable ALTER COLUMN _never DateTime NULL
go
IF EXISTS (SELECT * FROM sys.indexes WHERE object_id = OBJECT_ID(N'[dbo].[CustomizableTable]') AND name = N'idx__never') DROP INDEX idx__never ON CustomizableTable WITH ( ONLINE = OFF );
go
UPDATE CustomizableTable SET [_never] = '1/1/1900' WHERE [_never] IS NULL
go
SELECT * FROM sys.columns WHERE object_id = OBJECT_ID(N'CustomizableTable') AND name = '_gonna'
go
ALTER TABLE CustomizableTable ALTER COLUMN _gonna Money NULL
go
IF EXISTS (SELECT * FROM sys.indexes WHERE object_id = OBJECT_ID(N'[dbo].[CustomizableTable]') AND name = N'idx__gonna') DROP INDEX idx__gonna ON CustomizableTable WITH ( ONLINE = OFF );
go
UPDATE CustomizableTable SET [_gonna] = 0 WHERE [_gonna] IS NULL
go

(Closing the Transaction...?)

exec sp_cursorclose 180150003
go

After all that ado above, only the deletion of the column occurs. Everything before and after it in the transaction appears to be ignored, and there were no messages in the SQL Trace to indicate that something went wrong during the transaction.

A: 

The behavior you describe is allowed. How is the code making the schema changes? Building SQL on the fly and executing through an ADO Command? Or using ADOX?

If you have access to the database server, try running a SQL Profiler trace while testing the scenario you outlined. See if the trace logs any errors/rollbacks.

Brannon
Oh, it's building ADO commands. I'll try using the trace and report back!
EdgarVerona
Okay, used the trace, I didn't see anything unusual though. =( I posted the SQL statements that were picked up by the Profiler above.
EdgarVerona
+1  A: 

The code is using a server-side cursor, that's what those calls are for. The first set of calls is preparing/opening the cursor. Then fetching rows from the cursor. Finally closing the cursor. Those sprocs are analogous to the OPEN CURSOR, FETCH NEXT, CLOSE CURSOR T-SQL statements.

I'd have to take a closer look (which I will), but my guess is there is something going on with the server-side cursor, the encapsulating transaction, and the DDL.

Some more questions:

  1. Are you meaning to use server-side cursors in this case?
  2. Are the ADO Commands all using the same active connection?

Update:

I'm not exactly sure what's going on.

It looks like you're using server-side cursors so you can use Recordset.Update() to push changes back to the server, in addition to executing generated SQL statements to alter schema and update data in the dynamic table(s). Using the same connection, inside an explicit transaction.

I'm not sure what effect the cursor operations will have on the rest of the transaction, or vice-versa, and to be honest I'm surprised this isn't working.

I don't know how large of a change it would be, but I would recommend moving away from the server-side cursors and building the UPDATE statements for your table updates.

Sorry I couldn't be of more help.

BTW- I found the following information on the sp_cursor calls:

http://jtds.sourceforge.net/apiCursors.html

Brannon
Aye, I was reading in the documentation that in order to do transactions I couldn't use a Client-Side Cursor. And I also made sure that they were all using the same active connection as well.Thank you for helping out with this, it's been a weird problem for sure =(
EdgarVerona
Good info too about the stored procedures there... I had a feeling it was doing something like that, but I wasn't sure... I'd never actually done a Trace before today, so I've never seen those kinds of procedures being called under the hood. =) =)
EdgarVerona
No worries, you tried the best you could with the information given. I'm scratching my head on it aas well... Aye, I am using Recordset.Update right now for the actual row updates as you said... I'll try using a directly generated Update statement instead and see how that works.
EdgarVerona