what is the maximum objects supported by LINQ

Question

I am trying to use LINQ for certain operations. So there may be chance that millions of record will be used in LINQ.

So is it worth to use LINQ in this case? what is the maximum object limit to use LINQ?

I am reading records from database and check some conditions and store in List<Result>. Result is a class. Then performing LINQ query in List<Result> like grouping, counting etc. So there may be chance that min 50,000 records in List<Result>, so in this whether its better to go for LINQ (or) reinsert the records to db and perform the queries?

Answer 1

It depends what you are doing with it. If you are only using it to construct queries for something else to process, for example Fluent NHibernate, then linq will never see the objects themselves, so won't be a factor.

If you're using linq to objects with millions of objects, you might want to avoid loading all of the objects into memory at once. In which case, you'll want to only use lazy linq queries. Jon has a good overview of the issues in his article Just how lazy are you?

Answer 2

LINQ itself works against the IEnumerable types, so can work with any size collection (even infinite sequences, provided that the result is calculatable - e.g. you can take the first five figures from a infinite series of numbers, but not find out the maximum).

The real bottleneck and performance issues will depend on where you are getting your data from. For example, LINQ to SQL will try and generate a SQL query to push most of the work onto the database, whereas LINQ to objects will require all the information to be in-memory and will be much slower when processing millions of records.

Answer 3

If you have "millions of record" in your database, you of course don't want to return them all to your LINQ client to process. However, the same is true whatever technology you are using - Linq, Entity Framework, NHibernate, ADO.Net etc.

If you use LINQ sensibly (i.e. FILTER YOUR RESULTS) then the number of rows in the backing store is largely irrelevant.

Answer 4

LINQ AFAIK is about querying objects in a flexible and easy way.

Language Integrated Queries ... it just writes your SQL queries in code, a better way without you having to learn all the vernaculars of SQL.

As long as your querying logic is sound, LINQ will only make your life easier - no matter what the data size.

Please correct me if I am wrong!