Performing part of a IQueryable query and deferring the rest to Linq for Objects

Question

I have a Linq provider that sucessfully goes and gets data from my chosen datasource, but what I would like to do now that I have my filtered resultset, is allow Linq to Objects to process the rest of the Expression tree (for things like Joins, projection etc)

My thought was that I could just replace the expression constant that contains my IQueryProvider with the result-sets IEnumerable via an ExpressionVisitor and then return that new expression. Also return the IEnumerable's provider from my IQueryable...but this does not seem to work :-(

Any idea's?

Edit: Some good answers here, but given the form...

var qry = from c in MyProv.Table<Customer>()
          Join o in MyProv.Table<Order>() on c.OrderID equals o.ID
          select new 
          {
            CustID = c.ID,
            OrderID = o.ID
          }

In my provider I can easily get back the 2 resultsets from Customers and Orders, if the data was from a SQL source I would just construct and pass on the SQL Join syntax, but it this case the data is not from a SQL source so I need to do the join in code...but as I said I have the 2 result sets, and Linq to Objects can do a join...(and later the projection) it would be real nice to just substitute the Expression constants MyProv.Table<Customer> and MyProv.Table<Order> with List<Customer> and List<Order> and let a List<> provider process the expression...is that possible? how?

Answer 1

Unless I'm misunderstanding, I generally just add .ToArray() in the chain of linq methods at the point where I want the linq provider to execute.

For example (think Linq to SQL)

var result = datacontext.Table
   .Where(x => x.Prop == val)
   .OrderBy(x => x.Prop2)
   .ToArray()
   .Select(x => new {CoolProperty = x.Prop, OtherProperty = x.Prop2});

So through OrderBy() gets translated into SQL, but the Select() is LINQ to Objects.

Answer 2

Rob's answer is good, but forces full enumeration. You could cast to keep extension method syntax and lazy evaluation:

var res = ((IEnumerable<Foo>)dc.Foos
            .Where(x => x.Bla > 0))  // IQueryable
          .Where(y => y.Snag > 0)   // IEnumerable

Answer 3

Both of the previous answers work, but it reads better if you use AsEnumerable() to cast the IQueryable to IEnumerable:

// Using Bob's code...
var result = datacontext.Table
   .Where(x => x.Prop == val)
   .OrderBy(x => x.Prop2)
   .AsEnumerable()  //  <---- anything after this is done by LINQ to Objects
   .Select(x => new { CoolProperty = x.Prop, OtherProperty = x.Prop2 });

EDIT:

// ... or MichaelGG's
var res = dc.Foos
           .Where(x => x.Bla > 0)  // uses IQueryable provider
           .AsEnumerable()
           .Where(y => y.Snag > 0); // IEnumerable, uses LINQ to Objects

Answer 4

The kind of thing that I was after was replacing the Queryable<> constant in the expression tree with a concrete IEnumerable (or IQueryable via .AsQueryable()) result set...this is a complex topic that probably only makes any sense to Linq Provider writers who are knee deep in expression tree visitors etc.

I found a snippet on the msdn walkthrough that does something like what I am after, this gives me a way forward...

using System;
using System.Linq;
using System.Linq.Expressions;

namespace LinqToTerraServerProvider
{
    internal class ExpressionTreeModifier : ExpressionVisitor
    {
        private IQueryable<Place> queryablePlaces;

        internal ExpressionTreeModifier(IQueryable<Place> places)
        {
            this.queryablePlaces = places;
        }

        internal Expression CopyAndModify(Expression expression)
        {
            return this.Visit(expression);
        }

        protected override Expression VisitConstant(ConstantExpression c)
        {
            // Replace the constant QueryableTerraServerData arg with the queryable Place collection.
            if (c.Type == typeof(QueryableTerraServerData<Place>))
                return Expression.Constant(this.queryablePlaces);
            else
                return c;
        }
    }
}

Answer 5

If you implemented a Repository Pattern you could get away with just providing an IQueryable back and abstract away the table.

Example:

var qry = from c in MyProv.Repository<Customer>()
          Join o in MyProv.Repository<Order>() on c.OrderID equals o.ID
          select new 
          {
            CustID = c.ID,
            OrderID = o.ID
          }

and then just build your provider to model the IQueryable pattern in your Repository method just like this article illustrates.

This way, you can write all kinds of providers to use for whatever you need. You can have a LINQ 2 SQL provider, or write an in memory provider for your unit tests.

The Repository method for the LINQ 2 SQL provider would look something like this:

public IQueryable<T> Repository<T>() where T : class
{
    ITable table = _context.GetTable(typeof(T));
    return table.Cast<T>();
}