views:

157

answers:

1

I just got a code handed over to me. The code is written in C# and it inserts realtime data into database every second. The data is accumulated in time which makes the numbers big.

The data is updated within the second many times then at the end of the second result is taken and inserted.

We used to address the dataset rows directly within the second through the properties. For example many operations like this one 'datavaluerow.meanvalue += mean; could take place. we figured out that this is degrading the performance after running the profiler becuase of the internal casting done so we created 2d array of decimals on which the updates are carried out then the values are assigned to the datarows only at the end of the second. I ran a profiler and found out that it is still taking a lot of time (although less than the time spent accessing datarows frequently when added up).

The code that is exectued at the end of the second is as follows

public void UpdateDataRows(int tick)
{
  //ord
  //_table1Values is of type decimal[][]
  for (int i = 0; i < _table1Values.Length; i++)
  {
    _table1Values[i][(int)table1Enum.barDateTime] = tick;
    table1Row[i].ItemArray = _table1Values[i].Cast<object>().ToArray();

  }
  // this process is done for other 10 tables            
}

Is there a way to further improve this approach.

+3  A: 

One obvious question: why do you have a 2D array of decimals when you're only updating them with integers? Could you get away with an int[][] instead?

Next, why are you accessing (int)table1Enum.barDateTime on each iteration? Given that there's a conversion involved there, you may find it helps if you extract that out of the loop.

However, I suspect the majority of the time is going to be spent in _table1Values[i].Cast<object>().ToArray(). Do you really need to do that? Taking a copy of the decimal[] (or int[]) would be faster than boxing every value on every iteration on every call - and then creating another array.

Jon Skeet
We are thinking of changing the decimal to uint and of course this will improve the memory utilization and performance. however the time is taken in casting the decimal to object. ItemArray is of type object. How can I update the data rows without doing that
Mustafa A. Jabbar
@Mustafa: Just because `ItemArray` is of type `object` doesn't mean you've got to have an object *array*. Any array is an object itself. You need to consider whether to take a copy of the array, mind you.
Jon Skeet
still copying the array is taking the same time. The thing is the line is taking like 20 ms on average but we are repeating this process for other tables which is ending up around 300 ms! the delay is very critical in our data. I found this process to be a big bottleneck
Mustafa A. Jabbar
@Mustafa: Well do you actually *need* to copy the array? That's something I can't tell you - does it matter to your particular program whether you essentially have one array per row, and modify it directly, or whether it always has to be an independent copy?
Jon Skeet
@Jon: I am not sure if I quite get your question. Each row in the table contains data for some company, we update this data in 1 second duration and insert at the end of it. I don't care about the data structure used in my code as long as it will not violate the structure of our data tables.
Mustafa A. Jabbar
@Mustafa: Currently you are creating a *copy* of each array and setting a reference to that copy as `ItemArray`. An alternative would be to set `ItemArray` to refer directly to the original array. Changes in the array would be visible via both `ItemArray` and `table1Values`. As I say, I don't know enough about what you're doing with them to know whether that will work.
Jon Skeet
The arrays contain numbers that undergo some simple and complex arithmetic operations
Mustafa A. Jabbar
@Mustafa: I'm not sure how that's relevant to the question of whether taking a copy is necessary or not.
Jon Skeet
Also in your approach I will end up assigning numbers to objects frequently while calculating the data values. I am not sure if this will perform as good as assigning to integers or decimals especially that we are doing this a lot in the code.
Mustafa A. Jabbar
@Jon: no need to take a copy. the whole thing is just to avoid accessing the object array frequnctly. Object a = 1234543 takes more time than decimal a = 1234543m. I can't explain more
Mustafa A. Jabbar
@Mustafa: I'm not suggesting using an object array. I've suggested (as per my second comment) using an `int[]` or `decimal[]` array.
Jon Skeet
@Downvoter: Care to provide a reason?
Jon Skeet