Where to draw the line - is it possible to love LINQ too much?

Question

I recently found LINQ and love it. I find lots of occasions where use of it is so much more expressive than the longhand version but a colleague passed a comment about me abusing this technology which now has me second guessing myself. It is my perspective that if a technology works efficiently and the code is elegant then why not use it? Is that wrong? I could spend extra time writing out processes "longhand" and while the resulting code may be a few ms faster, it's 2-3 times more code and therefore 2-3 times more chance that there may be bugs.

Is my view wrong? Should I be writing my code out longhand rather than using LINQ? Isn't this what LINQ was designed for?

Edit: I was speaking about LINQ to objects, I don't use LINQ to XML so much and I have used LINQ to SQL but I'm not so enamoured with those flavours as LINQ to objects.

Answer 1

You're answering your own question by talking about writing 2-3 times more code for a few ms of performance. I mean, if your problem domain requires that speedup then yes, if not probably not. However, is it really only a few ms of performance or is it > 5% or > 10%. This is a value judgement based on the individual case.

Answer 2

LINQ's supposed to be used to make filtering, sorting, aggregating and manipulating data from various sources as intuitive and expressive as possible. I'd say, use it wherever you feel it's the tidiest, most expressive and most natural syntax for doing what it is you're trying to do, and don't feel guilty about it.

If you start humping the documentation, then it may be time to reconsider your position.

Answer 3

It's cases like these where it's important to remember the golden rules of optimization:

1. Don't Do It
2. For Experts: Don't do it yet

You should absolutely not worry about "abusing" linq unless you can indentify it explicitly as the cause of a performance problem

Answer 4

I have to agree with your view - if it's more efficient to write and elegant then what's a few milliseconds. Writing extra code gives more room for bugs to creep in and it's extra code that needs to be tested and most of all it's extra code to maintain. Think about the guy who's going to come in behind you and maintain your code - they'll thank you for writing elegant easy to read code long before they thank you for writing code that's a few ms faster!

Beware though, this cost of a few ms could be significant when you take the bigger picture into account. If that few milliseconds is part of a loop of thousands of repetitions, then the milliseconds add up fast.

Answer 5

Like anything, it can be abused. As long as you stay away from obvious poor decisions such as

var v = List.Where(...);
for(int i = 0; i < v.Count(); i++)
{...}

and understand how differed execution works, then it is most likely not going to be much slower than the longhand way. According to Anders Hejlsburg (C# architect), the C# compiler is not particularly good at optimizing loops, however it is getting much better at optimizing and parallelizing expression trees. In time, it may be more effective than a loop. The List<>'s ForEach version is actually as fast as a for loop, although I can't find the link that proves that.

P.S. My personal favorite is ForEach<>'s lesser known cousin IndexedForEach (utilizing extension methods)

List.IndexedForEach( (p,i) => 
 {
     if(i != 3)
        p.DoSomething(i);
 };

Answer 6

Its not possible to love Linq to Objects too much, it's a freaking awesome technology !

But seriously, anything that makes your code simple to read, simple to maintain and does the job it was intended for, then you would be silly not to use it as much as you can.

Answer 7

Yes you can love LINQ too much - Single Statement LINQ RayTracer

Where do you draw the line? I'd say use LINQ as much as it makes the code simpler and easier to read.

The moment the LINQ version becomes more difficult to understand then the non-LINQ version it's time to swap, and vice versa. EDIT: This mainly applies to LINQ-To-Objects as the other LINQ flavours have their own benefits.

Answer 8

Where to draw the line?

Well, we already know that it is a bad idea to implement your own quicksort in linq, at least compared to just using linq's orderby.

Answer 9

I've found that using LINQ has speed up my development and made it easier to avoid stupid mistakes that loops can introduce. I have had instances where the performance of LINQ was poor, but that was when I was using it to things like fetch data for an excel file from a tree structure that had millions of nodes.

Answer 10

While I see how there is a point of view that LINQ might make a statement harder to read, I think it is far outweighed by the fact that my methods are now strictly related to the problems that they are solving and not spending time either including lookup loops or cluttering classes with dedicated lookup functions.

It took a little while to get used to doing things with LINQ, since looping lookups, and the like, have been the main option for so long. I look at LINQ as just being another type of syntactic sugar that can do the same task in a more elegant way. Right now, I am still avoiding it in processing-heavy mission critical code - but that is just until the performance improves as LINQ evolves.

Answer 11

My only concern about LINQ is with its implementation of joins.

As I determined when trying to answer this question (and it's confirmed here), the code LINQ generates to perform joins is (necessarily, I guess) naive: for each item in the list, the join performs a linear search through the joined list to find matches.

Adding a join to a LINQ query essentially turns a linear-time algorithm into a quadratic-time algorithm. Even if you think premature optimization is the root of all evil, the jump from O(n) to O(n^2) should give you pause. (It's O(n^3) if you join through a joined item to another collection, too.)

It's relatively easy to work around this. For instance, this query:

var list = from pr in parentTable.AsEnumerable()
join cr in childTable.AsEnumerable() on cr.Field<int>("ParentID") equals pr.Field<int>("ID")
where pr.Field<string>("Value") == "foo"
select cr;

is analogous to how you'd join two tables in SQL Server. But it's terribly inefficient in LINQ: for every parent row that the where clause returns, the query scans the entire child table. (Even if you're joining on an unindexed field, SQL Server will build a hashtable to speed up the join if it can. That's a little outside LINQ's pay grade.)

This query, however:

string fk = "FK_ChildTable_ParentTable";
var list = from cr in childTable.AsEnumerable()
where cr.GetParentRow(fk).Field<string>("Value") == "foo"
select cr;

produces the same result, but it scans the child table once only.

If you're using LINQ to objects, the same issues apply: if you want to join two collections of any significant size, you're probably going to need to consider implementing a more efficient method to find the joined object, e.g.:

Dictionary<Foo, Bar> map = buildMap(foos, bars);
var list = from Foo f in foos
where map[f].baz == "bat"
select f;

Answer 12

LINQ can be like art. Keep using it to make the code beautiful.