What is the best way to hold an Entity Framework object graph in memory?

Question

Hi All,

I have a long running process that integrates over a collection of incoming Entities to work out if the incoming entity is different from the entity in persistent storage.

When I first wrote this I retrieved each entity from the database that matched the entity I was comparing against, I also had some includes to pull in referenced entities that I also needed to compare. This was incredibly slow because each compare involved a database query and then Entity Framework 'fix up'.

I am now using a query to pull in all the entities I need for my comparisons before I start the process. This lets me take the speed hit up front so I can significantly speed up my process. However the intial query still takes at least a minute or two to execute.

Dictionary<long, SomeEntity> someEntities = new SomeEntitiesInclude().ApplyTo(context.SomeEntities)
                                                        .Where(se => se.SomeRelatedEntity.ID == relatedEntityID)
                                                        .Where(se => se.SCDCurrent == true)
                                                        .OrderBy<SomeEntity, long>(se => se.SomeRelatedEntity.ID)
                                                        .ToDictionary<SomeEntity, long>(se => se.SomeRelatedEntity.ID);

The actual sql query produced from this only takes a couple of seconds to execute so I think most of the time is spent doing 'fix up' so the context can track changes.

The SomeEntitiesInclude object lets me use strongly typed include statements and is a sub class of the IncludeStratergy class developed by Alex James : http://blogs.msdn.com/alexj/archive/2009/07/25/tip-28-how-to-implement-include-strategies.aspx

Does any one have tips to speed up the execution of this?

Apologies for the long question.

James

Answer 1

There are multiple things which could be slowing this down. Fix up this one of them. It is easy to test whether fix up is the problem, by projecting onto a non-entity type. That removes fix up from the equation altogether. If this solves the problem, then it probably was fix up slowing things down, and we can involve the solution from there.

On the other hand, it might not solve the problem, because there other things which could slow down the execution of the query. The first thing I would look at is compilation of the query from a LINQ expression into SQL. You can segregate this step by using the CompiledQuery type. If producing the CompiledQuery takes a long time, but executing the resulting method takes very little time, then you have found the source of the performance issue.

Before doing any of this, however, I would probably make sure that your query is not unnecessarily complicated. Without knowing what you're doing, precisely, I cannot say if you actually need all of the fields you're returning. But if you need only a few fields from a few types, you would do well to project onto a non-entity type, rather than returning fully-materialized entities.