Help needed for optimizing linq data extraction

Question

I'm fetching data from all 3 tables at once to avoid network latency. Fetching the data is pretty fast, but when I loop through the results a lot of time is used

Int32[] arr = { 1 };
var query = from a in arr
            select new
            {
              Basket = from b in ent.Basket
                       where b.SUPERBASKETID == parentId
                       select new
                       {
                           Basket = b,
                           ObjectTypeId = 0, 
                           firstObjectId = "-1",
                       },

              BasketImage = from b in ent.Image
                            where b.BASKETID == parentId
                            select new
                            {
                                Image = b,
                                ObjectTypeId = 1, 
                                CheckedOutBy = b.CHECKEDOUTBY,
                                firstObjectId = b.FIRSTOBJECTID,
                                ParentBasket = (from parentBasket in ent.Basket
                                                where parentBasket.ID == b.BASKETID
                                                select parentBasket).ToList()[0],
                            },

              BasketFile = from b in ent.BasketFile
                           where b.BASKETID == parentId
                           select new
                           {
                               BasketFile = b,
                               ObjectTypeId = 2, 
                               CheckedOutBy = b.CHECKEDOUTBY,
                               firstObjectId = b.FIRSTOBJECTID,
                               ParentBasket = (from parentBasket in ent.Basket
                                               where parentBasket.ID == b.BASKETID
                                               select parentBasket),
                           }
            };

//Exception handling

var mixedElements = query.First();
ICollection<BasketItem> basketItems = new Collection<BasketItem>();

//Here 15 millis has been used
//only 6 elements were found

if (mixedElements.Basket.Count() > 0)
{
  foreach (var mixedBasket in mixedElements.Basket){}
}

if (mixedElements.BasketFile.Count() > 0)
{
  foreach (var mixedBasketFile in mixedElements.BasketFile){}
}

if (mixedElements.BasketImage.Count() > 0)
{
  foreach (var mixedBasketImage in mixedElements.BasketImage){}
}

//the empty loops takes 811 millis!!

Answer 1

You're using IEnumerable in the foreach loop. Implementations only have to prepare data when it's asked for. In this way, I'd suggest that the above code is accessing your data lazily -- that is, only when you enumerate the items (which actually happens when you call Count().)

Put a System.Diagnostics.Stopwatch around the call to Count() and see whether that's taking the bulk of the time you're seeing.

I can't comment further here because you don't specify the type of ent in your code sample.

Answer 2

Why are you bothering to check the counts before the foreach statements? If there are no results, the foreach will just finish immediately.

Your queries are actually all being deferred - they'll be executed as and when you ask for the data. Don't forget that your outermost query is a LINQ to Objects query: it's just returning the result of calling ent.Basket.Where(...).Select(...) etc... which doesn't actually execute the query.

Your plan to do all three queries in one go isn't actually working. However, by asking for the count separately, you may actually be executing each database query twice - once just getting the count and once for the results.

I strongly suggest that you get rid of the "optimizations" in this code which are making it much more complicated and slower than just writing the simplest code you can.

I don't know of any way of getting LINQ to SQL (or LINQ to EF) to execute multiple queries in a single call - but this approach certainly isn't going to do it.

One other minor hint which is irrelevant in this case, but can be useful in LINQ to Objects - if you want to find out whether there's any data in a collection, just use Any() instead of Count() > 0 - that way it can stop as soon as it's found anything.

Answer 3

Thank you for the answers, ill just correct the query back it's simplest version