I'm exporting data programatically from Excel to SQL Server 2005 using SqlBulkCopy. It works great, the only problem I have is that it doesn't preserve the row sequence i have in Excel file. I don't have a column to order by, I just want the records to be inserted in the same order they appear in the Excel Spreadsheet. I can't modify th...
Nightly, I need to fill a SQL Server 2005 table from an ODBC source with over 8 million records. Currently I am using an insert statement from linked server with syntax select similar to this: Insert Into SQLStagingTable from Select * from OpenQuery(ODBCSource, 'Select * from SourceTable') This is really inefficient and takes hours to...
I'm performing a bulk insert with an ADO.NET 2.0 SqlBulkCopy object from a C# method into a MS SQL 2005 database, using a database user with limited permissions. When I try to run the operation, I get the error message: Bulk copy failed. User does not have ALTER TABLE permission on table 'theTable'. ALTER TABLE permission is r...
I have a dataSet populated from Excel Sheet. I wanted to use SQLBulk Copy to Insert Records in Lead_Hdr table where LeadId is PK. I am having following error while executing this code The given ColumnMapping does not match up with any column in the source or destination string ConStr=ConfigurationManager.ConnectionStrings["ConSt...
I am using the SqlBulkCopy class to do a bulk insert into a SQLServer DB. Original size of the .mdf file associated with the DB is 1508 Mb. When I run it (on the same data of about 4 million records) with : BatchSize of 100000, the size of the .mdf grows to 1661 MB. BatchSize of 1000000, size of the .mdf grows to 1659 MB. Why this vari...
I am trying to copy a large Excel spreadsheet into the SQL Server database. I am opening an OldDbConnection to the Excel spreadsheet and reading everything from the [Sheet1$]. Then, I am using the OleDbCommand to get a IDataReader with the spreadsheet data. There are several cells in the Excel sheet with text contents of more than 256 ...
We have an in memory DataTable in a CLR procedure. After significant processing the DataTable has a lot of data that we need to load into a table in the database. Unfortunately, since we are using a context connection SqlBulkCopy will not work (throws error: The requested operation is not available on the context connection outline of c...
I've created a page where our partners can upload excel files with statistics using SQLBulkCopy. The file has multiple sheets and I need to add the name of the sheet into a column in the DB for each row. The names of the sheets will not change so hardcoding the names is ok. How can I solve this? ...
I have an ODBC Database (some third party DB) where I have a bunch of tables. I am able to use 'GetSchema' to get a list of tables in there. I am also abe to utilize SQLBulkCopy to copy Data from these tables to SQL Server tables (only when I have created a Destination table with the same structure in SQL Server). But here is the scenar...
I have done some research for "The bast way to insert huge data into DB with C#" then a lot of people just suggested me using SqlBulkCopy. After I tried it out and it really amazed me. Undoubtedly, SqlBulkCopy is very very fast. It seems that SqlBulkCopy is a perfect way to insert data (especially huge data). But why dont we use it at al...
I am trying to insert huge amount of data into SQL server. My destination table has an unique index called "Hash". I would like to replace my SqlDataAdapter implementation with SqlBulkCopy. In SqlDataAapter there is a property called "ContinueUpdateOnError", when set to true adapter.Update(table) will insert all the rows possible and ...
I'm trying to batch insert data into SQL 2008 using SqlBulkCopy. Here is my table: IF OBJECT_ID(N'statement', N'U') IS NOT NULL DROP TABLE [statement] GO CREATE TABLE [statement]( [ID] INT IDENTITY(1, 1) NOT NULL, [date] DATE NOT NULL DEFAULT GETDATE(), [amount] DECIMAL(14,2) NOT NULL, CONSTRAINT [PK_statement] PRIMARY KEY CLUSTE...
I'm not an expert with either System.Data nor SQL Server, but have a need to generate a large DataSet then use System.Data.SqlClient.SqlBulkCopy to store the results. The DataSet will consist of about 10 related tables. When SqlBulkCopy transfers the DataSet to SQL Server, my understanding is that rows from the various DataTables will ...
I am using SQLBulkCopy to move large amounts of data. I implemented the notification event to notify me every time a certain number of rows have been processed, but the OnSqlRowsCopied event does not fire when the job is completed. How do I get the total number of rows copied when the SQLBulkCopy writetoserver completes? ...
We have a need to update several tables that have parent/child relationships based on an Identity primary-key in the parent table, which is referred to by one or more child tables as a foreign key. Due to the high volume of data, we would like to build these tables in memory, then use SqlBulkCopy from C# to update the database en mass ...
I want to truncate a table before doing a SqlBulkCopy. Does SqlBulkCopy automatically truncate a table before copying the data? ...
I'm trying to transfer the data from SQLite to SQL Server. The schema of target and destination table are just the same: SQL Server: CREATE TABLE [dbo].[Shop] ( [ShopID] [int] IDENTITY(1,1) NOT NULL, [Name] [nvarchar](128) NOT NULL, [Url] [nvarchar](128) NOT NULL, PRIMARY KEY CLUSTERED ( [ShopID] ASC )) and SQLite: ...
I'm importing some data from a CSV, and it doesn't have all the fields my table does. I was making a DataTable with only the columns of my CSV, but I'm getting errors about converting from String to Boolean. None of the fields I'm adding are Boolean, so I'm assuming its a problem with the fields I left alone. Do I need to have a 1-1 m...
I just realized my caps was on from working in SQL Server, ha. Anyways, Im trying to write from serverName.databaseName.databaseInstanceName.TableName to C:\FileName.xml using bcp and I want to write only columns Col1 and Col2, could I get a little syntax help? I'm struggling over here. Thanks, Ted ...
If SqlBulkCopyOptions.CheckConstraints option is set for SqlBulkCopy insert, does it check the constraints separately after each record, or after all records are inserted? I have a check constraint that compares some columns to the parent row (through a UDF). I'm using MS Sql Server 2005. The documentation at http://msdn.microsoft.com/e...