tags:

views:

12125

answers:

40

Background: Over the next month, I'll be giving three talks about or at least including LINQ in the context of C#. I'd like to know which topics are worth giving a fair amount of attention to, based on what people may find hard to understand, or what they may have a mistaken impression of. I won't be specifically talking about LINQ to SQL or the Entity Framework except as examples of how queries can be executed remotely using expression trees (and usually IQueryable).

So, what have you found hard about LINQ? What have you seen in terms of misunderstandings? Examples might be any of the following, but please don't limit yourself!

  • How the C# compiler treats query expressions
  • Lambda expressions
  • Expression trees
  • Extension methods
  • Anonymous types
  • IQueryable
  • Deferred vs immediate execution
  • Streaming vs buffered execution (e.g. OrderBy is deferred but buffered)
  • Implicitly typed local variables
  • Reading complex generic signatures (e.g. Enumerable.Join)
+1  A: 

Lazy Loading.

Ryan Eastabrook
+168  A: 

Delayed execution

JaredPar
Righto - this is clearly the favourite amongst readers, which is the most important thing for this question. I'll add "buffering vs streaming" into the mix as well, as that's closely related - and often isn't discussed in as much detail as I'd like to see in books.
Jon Skeet
Really? I had it's lazy-loaded nature pointed out to me so many times while learning Linq, it was never an issue for me.
Adam Lassek
@ALassek, it really depends on where you learn LINQ from. I've presented for user groups where not a one knew than it was lazy evaluated.
JaredPar
Agree with ALassek. The MSDN documentation clearly states the lazy evaluation nature of LINQ. Maybe the real problem is the lazy programming nature of the developers... =)
Seiti
... especially when you realize that it applies to LINQ to objects and not just LINQ 2 SQL - when you see 10 web method calls to retrieve a list of items when you're already enumerating through that same list of items and you thought the list was already evaluated
Simon_Weaver
This should be deferred execution, right? Anders has mentioned this many times in his videos about LINQ in MSDN's Channel 9.
eriawan
@eriawan, delayed or deferred works :)
JaredPar
Knowing what the yield statement is and how it works is IMHO critical for a thorough understanding of LINQ.
peSHIr
+63  A: 

That there is more than just LINQ to SQL and the features are more than just a SQL parser embedded in the language.

smaclell
I'm sick of everyone thinking that :/
TraumaPony
Not everyone does! I still don't know what LINQ to SQL is, and I use LINQ all the damn time.
Robert Rossney
I get so annoyed when I try and explain something using LINQ and the other person just looks at me and says "ohhh I don't use LINQ for anything like that, only SQL" :(
Nathan W
Agreed, many people don't seem to understand that LINQ is a general purpose tool.
Matt Olenik
+40  A: 

I think the fact that a Lambda expression can resolve to both an expression tree and an anonymous delegate, so you can pass the same declarative lambda expression to both IEnumerable<T> extension methods and IQueryable<T> extension methods.

Tim Jarvis
Agreed. I'm a veteran and I just realized this implicit casting was taking place as I started writing my own QueryProvider
TheSoftwareJedi
+1  A: 

I think you should give more attention to the most commonly used features of LINQ in detail - Lambda expressions and Anonymous types, rather than wasting time on "hard to understand" stuff that is rarely used in real world programs.

I agree with the principle, but in reality almost all of the hard-to-understand bits *are* used frequently in real world programs - just without people really understanding them.
Jon Skeet
+15  A: 

I still have trouble with the "let" command (which I've never found a use for) and SelectMany (which I've used, but I'm not sure I've done it right)

James Curran
Any time you want to introduce a variable you would use a let statement. Think of a traditional loop where you are introducing variables within it and giving each variable a name to help the readability of code. Sometimes it is also nice where you have a let statement evaluating a function result, which you can then select and order by on without having to evaluate the result twice.
Rob Packwood
+30  A: 

In LINQ to SQL I constantly see people not understanding the DataContext, how it can be used and how it should be used. Too many people don't see the DataContext for what it is, a Unit of Work object, not a persistant object.

I've seen plenty of times where people are trying to singleton a DataContext/ session it/ etc rather than making a new time for each operation.

And then there's disposing of the DataContext before the IQueryable has been evaluated but that's more of a prople with people not understanding IQueryable than the DataContext.

The other concept I see a lot of confusion with is Query Syntax vs Expression Syntax. I will use which ever is the easiest at that point, often sticking with Expression Syntax. A lot of people still don't realise that they will produce the same thing in the end, Query is compiled into Expression after all.

Slace
Warning: Unit of work can be a small program with the data context as a singleton.
graffic
You should not use the DataContext in a singleton, it's not thread safe.
Slace
@Slace, not all programs are multitheaded, so it is OK to have the DataContext as a singleton in a lot of "desktop" software
Ian Ringrose
I got bitten by this (using DataContext as a singleton) when I did my first LINQ to SQL project. I don't think that the documentation and books make this obvious enough. Actually, I think the name could be improved, but I'm not sure how.
Roger Lipscombe
It took reading ScottGu's artivles on Linq multiple times to get this pounded in my head.
Evan Plaice
+11  A: 

Understanding when the abstraction among Linq providers leaks. Some things work on objects but not SQL (e.g., .TakeWhile). Some methods can get translated into SQL (ToUpper) while others can't. Some techniques are more efficient in objects where others are more effective in SQL (different join methods).

dpp
This is a very good point. It doesn't help that Intellisense will show you ALL of them and it will usually even compile. Then you blow up at runtime. I hope VS 2010 does a better job of showing relevant extension methods.
Jason Short
+21  A: 

I for one would sure like to know if I need to know what expression trees are, and why.

Robert Rossney
I think it's worth knowing what expression trees are and why they exist, but not the details of how to build them yourself. (They're a pain to build by hand, but the compiler will do a great job when converting a lambda expression.)
Jon Skeet
Actually, I was thinking about doing a few blog entries on expression trees (since I do "get" them). I find manipulating expression trees very useful...
Marc Gravell
However, I don't think they be helpful for Jon's talk(s) ;-p
Marc Gravell
I'll need to mention them briefly, but your blog entries would certainly be welcome :)
Jon Skeet
I just worry that expression trees are going to be like the yield statement: something that turned out to be incredibly valuable despite the fact that I didn't understand what it was for at first.
Robert Rossney
Marc Gravell I'd love to read your blog entries on the subject. Looking forward to it
Alexandre Brisebois
I have added a post to this chain with a link...
Marc Gravell
+15  A: 

I'm fairly new to LINQ. Here's the things I stumbled over in my first attempt

  • Combining several queries into one
  • Effectively debugging LINQ queries in Visual Studio.
Mark Heath
Debugging LINQ is a topic all by itself, and an important one. I think the greatest weakness of LINQ is that it lets you write blocks of arbitrarily complex logic that you can't step through.
Robert Rossney
these may be a good place to use LINQ pad
Maslow
+7  A: 

OK, due to demand, I've written up some of the Expression stuff. I'm not 100% happy with how blogger and LiveWriter have conspired to format it, but it'll do for now...

Anyway, here goes... I'd love any feedback, especially if there are areas where people want more information.

Here it is, like it or hate it...

Marc Gravell
+8  A: 

Couple of things.

  1. People thinking of Linq as Linq to SQL.
  2. Some people think that they can start replacing all foreach/logic with Linq queries without considering this performance implications.
Krishna Kumar
+1  A: 

Which is faster, inline Linq-to-Sql or Linq-to-Sql using Tsql Sprocs

... and are there cases where it's better to use server-side (Sproc) or client-side (inline Linq) queries.

+4  A: 

What does var represent when a query is executed?

Is it iQueryable, iSingleResult, iMultipleResult, or does it change based on the the implementation. There's some speculation about using (what appears to be) dynamic-typing vs the standard static-typing in C#.

AFAIK var always is the concrete class in question (even if it's an anonymous type) so it's *never* IQueryable, ISingleResult or anything beginning with 'I' (concrete classes that begin with 'I' need not apply).
Motti
+1  A: 

Comprehension syntax 'magic'. How does comprehension syntax gets translated into method calls and what method calls are chosen.

How does, for example:

from a in b
from c in d
where a > c
select new { a, c }

gets translated into method calls.

Pop Catalin
I did a little bit of this in the talk, but mostly just "this is the *sort* of thing the compiler does" - including "it doesn't know that the translation will call extension methods etc". The details are pretty complicated, of course...
Jon Skeet
(I did include a bit on transparent identifiers though, which is relevant to your example.)
Jon Skeet
I always find it easier to understand if I try to rethink it as a method chain: b.SelectMany(a => d, (a,c) => new { a=a, c=c }).Where(thing => thing.a > thing.c).Select(otherthing => new {a=otherthing.a, c=otherthing.c} )
JerKimball
+3  A: 

group by still makes my head spin.

Any confusion about deferred execution should be able to be resolved by stepping through some simple LINQ-based code and playing around in the watch window.

Richard Ev
I've found that implementing quite a bit of LINQ to Objects for fun really helps :) But yes, it is a bit confusing - certainly if I haven't done any LINQ for a while I have to go back to the signatures. Likewise "join" vs "join into" often gets me...
Jon Skeet
+1  A: 

For LINQ2SQL : Getting your head around some of the generated SQL and writing LINQ queries that translate to good (fast) SQL. This is part of the larger issue of knowing how to balance the declarative nature of LINQ queries with the realism that they need to execute fast in a known environment (SQL Server).

You can get a completely different SQL generated query by changing a tiny tiny thing in the LINQ code. Can be especially dangerous if you are creating an expression tree based on conditional statements (i.e. adding optional filtering criteria).

Simon_Weaver
+14  A: 
Slace
You might find this blog post interesting too: http://msmvps.com/blogs/jon_skeet/archive/2008/02/29/odd-query-expressions.aspx
Jon Skeet
Nice post Jon (I do subscribe to your blog, only recently though).
Slace
+1  A: 

I find it a bit disappointing that the query expression syntax only supports a subset of the LINQ functionality, so you cannot avoid chaining extension methods every now and then. E.g. the Distinct method cannot be called using the query expression syntax. To use the Distinct method you need to call the extension method. On the other hand the query expression syntax is very handy in many cases, so you don't want to skip that either.

A talk on LINQ could include some practical guidelines for when to prefer one syntax over the other and how to mix them.

Brian Rasmussen
Personally I'm glad that the query expression syntax doesn't include very many operators. The dot notation is fine when you need to use it, and this balance keeps C# still a *reasonably* small language. The query expression part of the spec is nice and short - I wouldn't want a really long section.
Jon Skeet
+2  A: 

As mentioned, lazy loading and deferred execution

How LINQ to Objects and LINQ to XML (IEnumerable) are different from LINQ to SQL(IQueryable)

HOW to build a Data Access Layer, Business Layer, and Presentation Layer with LINQ in all layers....and a good example.

Ash Machine
The first two I can do. I wouldn't like to try doing the third yet in a "this is the right way to do it" sense...
Jon Skeet
+1, until you pointed it out, I hadn't realized that LINQ-to-Objects and LINQ-to-XML were IEnumerable as opposed to LINQ-to-SQL as IQueryable, but it makes sense. Thanks!
Pretzel
+4  A: 

I don't know if it qualifies as misunderstood - but for me, simply unknown.

I was pleased to learn about DataLoadOptions and how I can control which tables are joined when I make a particular query.

See here for more info: MSDN: DataLoadOptions

Martin
+5  A: 

I think a great thing to cover in LINQ is how you can get yourself in trouble performance-wise. For instance, using LINQ's count as a loop condition is really, really not smart.

Steve
+4  A: 

I would say the most misunderstood (or should that be non-understood?) aspect of LINQ is IQueryable and custom LINQ providers.

I have been using LINQ for a while now and am completely comfortable in the IEnumerable world, and can solve most problems with LINQ.

But when I started to look at and read about IQueryable, and Expressions and custom linq providers it made my head spin. Take a look at how LINQ to SQL works if you want to see some pretty complex logic.

I look forward to understanding that aspect of LINQ...

Schneider
+1  A: 

This is of course not 'the most hardest' but just something to add to the list :

ThenBy() extension method

Without looking at its implementation I'm initially puzzled as to how it works. Everyone understands just fine how comma separated sort fields work in SQL - but on face value I'm skeptical that ThenBy is going to do what I really want it to do. How can it 'know' what the previous sort field was - it seems like it ought to.

I'm off to research it now...

Simon_Weaver
The trick is that ThenBy is an extension method on IOrderedEnumerable (or IOrderedQueryable) rather than just IEnumerable/IQueryable. You can download my (very naive!) implementation from my talks page: http://csharpindepth.com/Talks.aspx - see "LINQ to Objects in 60 minutes"
Jon Skeet
+3  A: 

As most people said, i think the most misunderstood part is assuming LINQ is a just a replacement for T-SQL. My manager who considers himself as a TSQL guru would not let us use LINQ in our project and even hates MS for releasing such a thing!!!

HashName
Too many people use it as a replacement for TSQL. Most of them have never heard of an execution plan.
erikkallen
+1 because I agree with your manager, at least insofar as allowing LINQ to SQL in any project. LINQ to Objects is another matter entirely.
Chris Lively
+14  A: 

I think the misunderstood part of LINQ is that it is a language extension, not a database extension or construct.

LINQ is so much more than LINQ to SQL.

Now that most of us have used LINQ on collections, we will NEVER go back!

LINQ is the single most significant feature to .NET since Generics in 2.0, and Anonymous Types in 3.0.

And now that we have Lambda's, I can't wait for parallel programming!

Chris
+56  A: 

I know the deferred execution concept should be beaten into me by now, but this example really helped me get a practical grasp of it:

static void Linq_Deferred_Execution_Demo()
{
    List<String> items = new List<string> { "Bob", "Alice", "Trent" };

    var results = from s in items select s;

    Console.WriteLine("Before add:");
    foreach (var result in results)
    {
        Console.WriteLine(result);
    }

    items.Add("Mallory");

    //
    //  Enumerating the results again will return the new item, even
    //  though we did not re-assign the Linq expression to it!
    //

    Console.WriteLine("\nAfter add:");
    foreach (var result in results)
    {
        Console.WriteLine(result);
    }
}

The above code returns the following:

Before add:
Bob
Alice
Trent

After add:
Bob
Alice
Trent
Mallory
DSO
+34  A: 

Big O notation. LINQ makes it incredibly easy to write O(n^4) algorithms without realizing it, if you don't know what you do.

erikkallen
Can anyone clarify on this?
Martinho Fernandes
How about an example?
hughdbrown
As far as an example, maybe he means the fact that it is very easy to have a Select clause contain many Sum() operators with each of them causing another pass over the entire recordset.
Rob Packwood
In fact it might even be worth going through what big O notation is and why it's important, as well as some examples of inefficient resulting queries. I think that's what the original poster was suggesting, but I thought I'd mention it anyways.--EDIT: just realised that this post was 1.5 years old :-)
zcrar70
+6  A: 

Some of the error messages, especially from LINQ to SQL can be pretty confusing. grin

I've been bitten by the deferred execution a couple of times like everyone else. I think the most confusing thing for me has been the SQL Server Query Provider and what you can and can't do with it.

I'm still amazed by the fact you can't do a Sum() on a decimal/money column that's sometimes empty. Using DefaultIfEmpty() just won't work. :(

Per Erik Stendahl
Shold be prette easy to slap a Where on that query to make sum work
Esben Skov Pedersen
A: 

Something i bet almost on one knows: you can use inline ifs in a linq query. Something like this:

var result = from foo in bars where (
    ((foo.baz != null) ? foo.baz : false) &&
    foo.blah == "this")
    select foo;

I would suppose you can insert lambdas as well although i haven't tried.

RCIX
That's just a conditional expression - why *wouldn't* you be able to use that?
Jon Skeet
I was of the (probably mistaken) impression that that is like a regular if: you wouldn't see any imbedding of regular ifs like that now would you? or i could be wrong...
RCIX
If is a statement and the conditional operator is an expression. They are different forms of the same concept of branching. In this case, though, you could do "foo.baz ?? false" and use the null coalescing operator :-)
Bryan Watts
I think more people know the ternary operator than the opposite..
devoured elysium
A: 

Detached object trees.

Anton Kutepov
+2  A: 

Compiled Queries

The fact that you can't chain IQueryable because they are method calls (while still nothing else but SQL translateable!) and that it is almost impossible to work around it is mindboggling and creates a huge violation of DRY. I need my IQueryable's for ad-hoc in which I don't have compiled queries (I only have compiled queries for the heavy scenarios), but in compiled queries I can't use them and instead need to write regular query syntax again. Now I'm doing the same subqueries in 2 places, need to remember to update both if something changes, and so forth. A nightmare.

Alex
+13  A: 

Took me way too long to realize that many LINQ extension methods such as Single(), SingleOrDefault() etc have overloads that take lambdas.

You can do :

Single(x => x.id == id)

and don't need to say this - which some bad tutorial got me in the habit of doing

Where(x => x.id == id).Single()
Simon_Weaver
+1, very nice. I'll keep it in mind.
Pretzel
+2  A: 

How easy it is to nest a loop is something I don't think everyone understands.

For example:

from outerloopitem in outerloopitems
from innerloopitem in outerloopitem.childitems
select outerloopitem, innerloopitem
Rob Packwood
+1, whoa. that's pretty powerful.
Pretzel
A: 

As most people said, i think the most misunderstood part is assuming LINQ is a just a replacement for T-SQL. My manager who considers himself as a TSQL guru would not let us use LINQ in our project and even hates MS for releasing such a thing!!!

stackuser1
A: 

The fact that you can't chain IQueryable because they are method calls (while still nothing else but SQL translateable!) and that it is almost impossible to work around it is mindboggling and creates a huge violation of DRY. I need my IQueryable's for ad-hoc in which I don't have compiled queries (I only have compiled queries for the heavy scenarios), but in compiled queries I can't use them and instead need to write regular query syntax again. Now I'm doing the same subqueries in 2 places, need to remember to update both if something changes, and so forth. A nightmare.

stackuser2
A: 

dynamic queries can get a bit hairy.

Prof Plum
+1  A: 

That IQueryable accept both, Expression<Func<T1, T2, T3, ...>> and Func<T1, T2, T3, ...>, without giving a hint about performance degradation in 2nd case.

Here is code example, that demonstrates what I mean:

[TestMethod]
public void QueryComplexityTest()
{
    var users = _dataContext.Users;

    Func<User, bool>                funcSelector =       q => q.UserName.StartsWith("Test");
    Expression<Func<User, bool>>    expressionSelector = q => q.UserName.StartsWith("Test");

    // Returns IEnumerable, and do filtering of data on client-side
    IQueryable<User> func = users.Where(funcSelector).AsQueryable();
    // Returns IQuerible and do filtering of data on server side
    // SELECT ... FROM [dbo].[User] AS [t0] WHERE [t0].[user_name] LIKE @p0
    IQueryable<User> exp = users.Where(expressionSelector);
}
Valera Kolupaev
Can you explain? I'm not following...
Pretzel
@Pretzel I have added code example, that demonstrate my issue.
Valera Kolupaev
A: 

Transactions (without using TransactionScope)

Naeem Sarfraz
+2  A: 

I think the #1 misconception about LINQ to SQL is that you STILL HAVE TO KNOW SQL in order to make effective use of it.

Another misunderstood thing about Linq to Sql is that you still have to lower your database security to the point of absurdity in order to make it work.

A third point is that using Linq to Sql along with Dynamic classes (meaning the class definition is created at runtime) causes a tremendous amount of just-in-time compiling. Which can absolutely kill performance.

Chris Lively
It is very beneficial to already know SQL, however. Some SQL that is emitted by Linq to SQL (and other ORM's) can be downright dubious, and knowing SQL helps diagnose such problems. Also, Linq to SQL can make use of stored procedures.
Robert Harvey