What's the hardest or most misunderstood aspect of LINQ?

Question

Background: Over the next month, I'll be giving three talks about or at least including LINQ in the context of C#. I'd like to know which topics are worth giving a fair amount of attention to, based on what people may find hard to understand, or what they may have a mistaken impression of. I won't be specifically talking about LINQ to SQL or the Entity Framework except as examples of how queries can be executed remotely using expression trees (and usually IQueryable).

So, what have you found hard about LINQ? What have you seen in terms of misunderstandings? Examples might be any of the following, but please don't limit yourself!

• How the C# compiler treats query expressions
• Lambda expressions
• Expression trees
• Extension methods
• Anonymous types
• IQueryable
• Deferred vs immediate execution
• Streaming vs buffered execution (e.g. OrderBy is deferred but buffered)
• Implicitly typed local variables
• Reading complex generic signatures (e.g. Enumerable.Join)

Answer 1

Lazy Loading.

Answer 2

Delayed execution

Answer 3

That there is more than just LINQ to SQL and the features are more than just a SQL parser embedded in the language.

Answer 4

I think the fact that a Lambda expression can resolve to both an expression tree and an anonymous delegate, so you can pass the same declarative lambda expression to both IEnumerable<T> extension methods and IQueryable<T> extension methods.

Answer 5

I think you should give more attention to the most commonly used features of LINQ in detail - Lambda expressions and Anonymous types, rather than wasting time on "hard to understand" stuff that is rarely used in real world programs.

Answer 6

I still have trouble with the "let" command (which I've never found a use for) and SelectMany (which I've used, but I'm not sure I've done it right)

Answer 7

In LINQ to SQL I constantly see people not understanding the DataContext, how it can be used and how it should be used. Too many people don't see the DataContext for what it is, a Unit of Work object, not a persistant object.

I've seen plenty of times where people are trying to singleton a DataContext/ session it/ etc rather than making a new time for each operation.

And then there's disposing of the DataContext before the IQueryable has been evaluated but that's more of a prople with people not understanding IQueryable than the DataContext.

The other concept I see a lot of confusion with is Query Syntax vs Expression Syntax. I will use which ever is the easiest at that point, often sticking with Expression Syntax. A lot of people still don't realise that they will produce the same thing in the end, Query is compiled into Expression after all.

Answer 8

Understanding when the abstraction among Linq providers leaks. Some things work on objects but not SQL (e.g., .TakeWhile). Some methods can get translated into SQL (ToUpper) while others can't. Some techniques are more efficient in objects where others are more effective in SQL (different join methods).

Answer 9

I for one would sure like to know if I need to know what expression trees are, and why.

Answer 10

I'm fairly new to LINQ. Here's the things I stumbled over in my first attempt

• Combining several queries into one
• Effectively debugging LINQ queries in Visual Studio.

Answer 11

OK, due to demand, I've written up some of the Expression stuff. I'm not 100% happy with how blogger and LiveWriter have conspired to format it, but it'll do for now...

Anyway, here goes... I'd love any feedback, especially if there are areas where people want more information.

Here it is, like it or hate it...

Answer 12

Couple of things.

1. People thinking of Linq as Linq to SQL.
2. Some people think that they can start replacing all foreach/logic with Linq queries without considering this performance implications.

Answer 13

Which is faster, inline Linq-to-Sql or Linq-to-Sql using Tsql Sprocs

... and are there cases where it's better to use server-side (Sproc) or client-side (inline Linq) queries.

Answer 14

What does var represent when a query is executed?

Is it iQueryable, iSingleResult, iMultipleResult, or does it change based on the the implementation. There's some speculation about using (what appears to be) dynamic-typing vs the standard static-typing in C#.

Answer 15

Comprehension syntax 'magic'. How does comprehension syntax gets translated into method calls and what method calls are chosen.

How does, for example:

from a in b
from c in d
where a > c
select new { a, c }

gets translated into method calls.

Answer 16

group by still makes my head spin.

Any confusion about deferred execution should be able to be resolved by stepping through some simple LINQ-based code and playing around in the watch window.

Answer 17

For LINQ2SQL : Getting your head around some of the generated SQL and writing LINQ queries that translate to good (fast) SQL. This is part of the larger issue of knowing how to balance the declarative nature of LINQ queries with the realism that they need to execute fast in a known environment (SQL Server).

You can get a completely different SQL generated query by changing a tiny tiny thing in the LINQ code. Can be especially dangerous if you are creating an expression tree based on conditional statements (i.e. adding optional filtering criteria).

Answer 18

I find it a bit disappointing that the query expression syntax only supports a subset of the LINQ functionality, so you cannot avoid chaining extension methods every now and then. E.g. the Distinct method cannot be called using the query expression syntax. To use the Distinct method you need to call the extension method. On the other hand the query expression syntax is very handy in many cases, so you don't want to skip that either.

A talk on LINQ could include some practical guidelines for when to prefer one syntax over the other and how to mix them.

Answer 19

As mentioned, lazy loading and deferred execution

How LINQ to Objects and LINQ to XML (IEnumerable) are different from LINQ to SQL(IQueryable)

HOW to build a Data Access Layer, Business Layer, and Presentation Layer with LINQ in all layers....and a good example.

Answer 20

I don't know if it qualifies as misunderstood - but for me, simply unknown.

I was pleased to learn about DataLoadOptions and how I can control which tables are joined when I make a particular query.

See here for more info: MSDN: DataLoadOptions

Answer 21

I think a great thing to cover in LINQ is how you can get yourself in trouble performance-wise. For instance, using LINQ's count as a loop condition is really, really not smart.

Answer 22

I would say the most misunderstood (or should that be non-understood?) aspect of LINQ is IQueryable and custom LINQ providers.

I have been using LINQ for a while now and am completely comfortable in the IEnumerable world, and can solve most problems with LINQ.

But when I started to look at and read about IQueryable, and Expressions and custom linq providers it made my head spin. Take a look at how LINQ to SQL works if you want to see some pretty complex logic.

I look forward to understanding that aspect of LINQ...

Answer 23

This is of course not 'the most hardest' but just something to add to the list :

ThenBy() extension method

Without looking at its implementation I'm initially puzzled as to how it works. Everyone understands just fine how comma separated sort fields work in SQL - but on face value I'm skeptical that ThenBy is going to do what I really want it to do. How can it 'know' what the previous sort field was - it seems like it ought to.

I'm off to research it now...

Answer 24

As most people said, i think the most misunderstood part is assuming LINQ is a just a replacement for T-SQL. My manager who considers himself as a TSQL guru would not let us use LINQ in our project and even hates MS for releasing such a thing!!!

Answer 25

I think the misunderstood part of LINQ is that it is a language extension, not a database extension or construct.

LINQ is so much more than LINQ to SQL.

Now that most of us have used LINQ on collections, we will NEVER go back!

LINQ is the single most significant feature to .NET since Generics in 2.0, and Anonymous Types in 3.0.

And now that we have Lambda's, I can't wait for parallel programming!

Answer 26

I know the deferred execution concept should be beaten into me by now, but this example really helped me get a practical grasp of it:

static void Linq_Deferred_Execution_Demo()
{
    List<String> items = new List<string> { "Bob", "Alice", "Trent" };

    var results = from s in items select s;

    Console.WriteLine("Before add:");
    foreach (var result in results)
    {
        Console.WriteLine(result);
    }

    items.Add("Mallory");

    //
    //  Enumerating the results again will return the new item, even
    //  though we did not re-assign the Linq expression to it!
    //

    Console.WriteLine("\nAfter add:");
    foreach (var result in results)
    {
        Console.WriteLine(result);
    }
}

The above code returns the following:

Before add:
Bob
Alice
Trent

After add:
Bob
Alice
Trent
Mallory

Answer 27

Big O notation. LINQ makes it incredibly easy to write O(n^4) algorithms without realizing it, if you don't know what you do.

Answer 28

Some of the error messages, especially from LINQ to SQL can be pretty confusing. grin

I've been bitten by the deferred execution a couple of times like everyone else. I think the most confusing thing for me has been the SQL Server Query Provider and what you can and can't do with it.

I'm still amazed by the fact you can't do a Sum() on a decimal/money column that's sometimes empty. Using DefaultIfEmpty() just won't work. :(

Answer 29

Something i bet almost on one knows: you can use inline ifs in a linq query. Something like this:

var result = from foo in bars where (
    ((foo.baz != null) ? foo.baz : false) &&
    foo.blah == "this")
    select foo;

I would suppose you can insert lambdas as well although i haven't tried.

Answer 30

Detached object trees.

Answer 31

Compiled Queries

The fact that you can't chain IQueryable because they are method calls (while still nothing else but SQL translateable!) and that it is almost impossible to work around it is mindboggling and creates a huge violation of DRY. I need my IQueryable's for ad-hoc in which I don't have compiled queries (I only have compiled queries for the heavy scenarios), but in compiled queries I can't use them and instead need to write regular query syntax again. Now I'm doing the same subqueries in 2 places, need to remember to update both if something changes, and so forth. A nightmare.

Answer 32

Took me way too long to realize that many LINQ extension methods such as Single(), SingleOrDefault() etc have overloads that take lambdas.

You can do :

Single(x => x.id == id)

and don't need to say this - which some bad tutorial got me in the habit of doing

Where(x => x.id == id).Single()

Answer 33

How easy it is to nest a loop is something I don't think everyone understands.

For example:

from outerloopitem in outerloopitems
from innerloopitem in outerloopitem.childitems
select outerloopitem, innerloopitem

Answer 34

As most people said, i think the most misunderstood part is assuming LINQ is a just a replacement for T-SQL. My manager who considers himself as a TSQL guru would not let us use LINQ in our project and even hates MS for releasing such a thing!!!

Answer 35

The fact that you can't chain IQueryable because they are method calls (while still nothing else but SQL translateable!) and that it is almost impossible to work around it is mindboggling and creates a huge violation of DRY. I need my IQueryable's for ad-hoc in which I don't have compiled queries (I only have compiled queries for the heavy scenarios), but in compiled queries I can't use them and instead need to write regular query syntax again. Now I'm doing the same subqueries in 2 places, need to remember to update both if something changes, and so forth. A nightmare.

Answer 36

dynamic queries can get a bit hairy.

Answer 37

That IQueryable accept both, Expression<Func<T1, T2, T3, ...>> and Func<T1, T2, T3, ...>, without giving a hint about performance degradation in 2nd case.

Here is code example, that demonstrates what I mean:

[TestMethod]
public void QueryComplexityTest()
{
    var users = _dataContext.Users;

    Func<User, bool>                funcSelector =       q => q.UserName.StartsWith("Test");
    Expression<Func<User, bool>>    expressionSelector = q => q.UserName.StartsWith("Test");

    // Returns IEnumerable, and do filtering of data on client-side
    IQueryable<User> func = users.Where(funcSelector).AsQueryable();
    // Returns IQuerible and do filtering of data on server side
    // SELECT ... FROM [dbo].[User] AS [t0] WHERE [t0].[user_name] LIKE @p0
    IQueryable<User> exp = users.Where(expressionSelector);
}

Answer 38

Transactions (without using TransactionScope)

Answer 39

I think the #1 misconception about LINQ to SQL is that you STILL HAVE TO KNOW SQL in order to make effective use of it.

Another misunderstood thing about Linq to Sql is that you still have to lower your database security to the point of absurdity in order to make it work.

A third point is that using Linq to Sql along with Dynamic classes (meaning the class definition is created at runtime) causes a tremendous amount of just-in-time compiling. Which can absolutely kill performance.