views:

39

answers:

2

HI Everyone,

I have SSIS packages which runs on nightly basis to move data from AS400 to SQL Server 2005. From last couple of months my packages consuming more time to complete as a scheduled job. If I run same packages manually they are completing normally . In all my ssis packages i am truncating the data in sql server then Bulk inserting data in those tables. Even I have lot of non clustered indexes on those tables. Do I need to drop all indexes on those tables before executing bulk insert SSIS packages and once the package is succeeded I have to create Non clustered indexes so that the performance and time consuming constraint of the packages can be solved. I am not sure this would be the problem ,Please guide me to solve this problem

A: 
From last couple of months my packages consuming more time to complete 
as a scheduled job. In all my ssis packages i am truncating the data in sql 
server then Bulk inserting data in those tables

I assume that over a period of time, the amount of data you are needing to bulk insert will keep on growing with this design? That might explain why the job might be becoming progressively slower.

Even I have lot of non clustered indexes on those tables. Do I need to drop all 
indexes on those tables before executing bulk insert SSIS packages and once the 
package is succeeded I have to create Non clustered indexes so that the 
performance and time consuming constraint of the packages can be solved

The indexes will definitely slow down the insert of the data. However, it could be a lot more than just your non-clustered indexes. Depending on the values of the columns used to make up the clustered index i.e primary key, it could be that the clustered index slowing down the inserts. This may depend on the ordering of the data that you are inserting and could be something that could be controlled.

As you have already highlighted, you could try out dropping and re-creating the indexes post insert. However, you have to factor in the time taken to re-create the indexes and see if that additional time results in enough of a performance improvement for you.

In general, i would suggest that you first log the amount of time taken in each step of your DTS to really pinpoint which step is the slowest and then decide how it can be improved. Currently, without data, you are really shooting in the dark

InSane
1. ya data has been grown from last one month but same package which takes 6 hours to complete as a job is completing in 2.5 hours in BIDS when i run manually.I am really unable to figure out why such a huge differnce beteween the 2 runs.
Mohammed Mushtaq
@Mohammed Mushtaq - Could it be because of the timing? Have you checked what other activities / jobs / transactions / processing is happening in the database at the time this job is scheduled? Something else MAY be locking up the row/ table's in question? Or it could just be the load on the DB at that point in time..Have u tried running the job at another time / day to see how it does?
InSane
this is a newly formatted server and nothing big tasks are running on it except my packages. ya i tried to run during the day time same issue i faced. since it is running during night time there is no point of any load . because in day time users use this server for MIS reports.
Mohammed Mushtaq
I think you will need to do some detailed logging of timings of each step + use the profiler to see exactly where the difference between the BIMS / Scheduled job is coming from! All out of ideas from my end!!
InSane
A: 

I have got good speeds of insert using OLEDB destination, with Data access mode=table or view fast load

You may play with the options of table lock, check constrains etc.

Saurabh Kumar
daer friend i have already set the option to fast load thank you for your reply
Mohammed Mushtaq