How can I write a clean Repository without exposing IQueryable to the rest of my application?

Question

So, I've read all the Q&A's here on SO regarding the subject of whether or not to expose IQueryable to the rest of your project or not (see here, and here), and I've ultimately decided that I don't want to expose IQueryable to anything but my Model. Because IQueryable is tied to certain persistence implementations I don't like the idea of locking myself into this. Similarly, I'm not sure how good I feel about classes further down the call chain modifying the actual query that aren't in the repository.

So, does anyone have any suggestions for how to write a clean and concise Repository without doing this? One problem I see, is my Repository will blow up from a ton of methods for various things I need to filter my query off of.

Having a bunch of:

IEnumerable GetProductsSinceDate(DateTime date);  
IEnumberable GetProductsByName(string name);  
IEnumberable GetProductsByID(int ID);

If I was allowing IQueryable to be passed around I could easily have a generic repository that looked like:

public interface IRepository<T> where T : class
{
    T GetById(int id);
    IQueryable<T> GetAll();
    void InsertOnSubmit(T entity);
    void DeleteOnSubmit(T entity);
    void SubmitChanges();
}

However, if you aren't using IQueryable then methods like GetAll() aren't really practical since lazy evaluation won't be taking place down the line. I don't want to return 10,000 records only to use 10 of them later.

What is the answer here? In Conery's MVC Storefront he created another layer called the "Service" layer which received IQueryable results from the respository and was responsible for applying various filters.

Is this what I should do, or something similar? Have my repository return IQueryable but restrict access to it by hiding it behind a bunch of filter classes like GetProductByName, which will return a concrete type like IList or IEnumerable?

Answer 1

Rob's method really doesn't solve your core problem, and that is not wanting to write individual methods for each type of query you'd like to run, and unfortunately if you're not using IQueryable then that is what you are left with.

Sure the methods might be in the "service" layer, but it still means having to write "GetProductsByName, GetProductsByDate"...

The other method is something like:

GetProducts(QueryObject);

This might give you some benefit over using IQueryable in that you can constrain what is returned.

Answer 2

hmm.. I solved this in many ways depending on the type of ORM i use.
The main idea is to have one repository base class and one query method that takes so many parameters indicating all possible where/orderby/expand|include/paging/etc options.

Here is a quick and dirty sample using LINQ to NHibernate (of course the entire repository should be implementation detail):

public class RepositoryBase
    {
        private ISession Session;

        public RepositoryBase()
        {
            Session = SessionPlaceHolder.Session;
        }



        public TEntity[] GetPaged<TEntity>(IEnumerable<Expression<Func<TEntity, bool>>> filters,
            IEnumerable<Expression<Func<TEntity, object>>> relatedObjects,
            IEnumerable<Expression<Func<TEntity, object>>> orderCriterias,
            IEnumerable<Expression<Func<TEntity, object>>> descOrderCriterias,
            int pageNumber, int pageSize, out int totalPages)
        {
            INHibernateQueryable<TEntity> nhQuery = Session.Linq<TEntity>();

            if (relatedObjects != null)
                foreach (var relatedObject in relatedObjects)
                {
                    if (relatedObject == null) continue;
                    nhQuery = nhQuery.Expand(relatedObject);
                }

            IQueryable<TEntity> query = nhQuery;

            if (filters != null)
                foreach (var filter in filters)
                {
                    if (filter == null) continue;
                    query = query.Where(filter);
                }

            bool pagingEnabled = pageSize > 0;

            if (pagingEnabled)
                totalPages = (int) Math.Ceiling((decimal) query.Count()/(decimal) pageSize);
            else
                totalPages = 1;

            if (orderCriterias != null)
                foreach (var orderCriteria in orderCriterias)
                {
                    if (orderCriteria == null) continue;
                    query = query.OrderBy(orderCriteria);
                }

            if (descOrderCriterias != null) 
                foreach (var descOrderCriteria in descOrderCriterias)
                {
                    if (descOrderCriteria == null) continue;
                    query = query.OrderByDescending(descOrderCriteria);
                }

            if (pagingEnabled)
                query = query.Skip(pageSize*(pageNumber - 1)).Take(pageSize);

            return query.ToArray();
        }
    }

Normally you'll want to add many chaining overloads as shortcuts when you don't need paging for example, etc..

Here is another dirty one. Sorry I'm not sure if I can expose the final ones. Those were drafts and are OK to show:

using Context = Project.Services.Repositories.EntityFrameworkContext;
using EntitiesContext = Project.Domain.DomainSpecificEntitiesContext;    
namespace Project.Services.Repositories
{
    public class EntityFrameworkRepository : IRepository
    {
        #region IRepository Members

        public bool TryFindOne<T>(Expression<Func<T, bool>> filter, out T result)
        {
            result = Find(filter, null).FirstOrDefault();

            return !Equals(result, default(T));
        }

        public T FindOne<T>(Expression<Func<T, bool>> filter)
        {
            T result;
            if (TryFindOne(filter, out result))
                return result;

            return default(T);
        }

        public IList<T> Find<T>() where T : class, IEntityWithKey
        {
            int count;
            return new List<T>(Find<T>(null, null, 0, 0, out count));
        }

        public IList<T> Find<T>(Expression<Func<T, bool>> filter, Expression<Func<T, object>> sort)
        {
            int count;
            return new List<T>(Find(filter, sort, 0, 0, out count));
        }

        public IEnumerable<T> Find<T>(Expression<Func<T, bool>> filter, Expression<Func<T, object>> sort, int pageSize,
                                      int pageNumber, out int count)
        {
            return ExecuteQuery(filter, sort, pageSize, pageNumber, out count) ?? new T[] {};
        }

        public bool Save<T>(T entity)
        {
            var contextSource = new EntityFrameworkContext();

            EntitiesContext context = contextSource.Context;

            EntityKey key = context.CreateEntityKey(GetEntitySetName(entity.GetType()), entity);

            object originalItem;
            if (context.TryGetObjectByKey(key, out originalItem))
            {
                context.ApplyPropertyChanges(key.EntitySetName, entity);
            }
            else
            {
                context.AddObject(GetEntitySetName(entity.GetType()), entity);
                //Attach(context, entity);
            }

            return context.SaveChanges() > 0;
        }

        public bool Delete<T>(Expression<Func<T, bool>> filter)
        {
            var contextSource = new EntityFrameworkContext();

            EntitiesContext context = contextSource.Context;

            int numberOfObjectsFound = 0;
            foreach (T entity in context.CreateQuery<T>(GetEntitySetName(typeof (T))).Where(filter))
            {
                context.DeleteObject(entity);
                ++numberOfObjectsFound;
            }

            return context.SaveChanges() >= numberOfObjectsFound;
        }

        #endregion

        protected IEnumerable<T> ExecuteQuery<T>(Expression<Func<T, bool>> filter, Expression<Func<T, object>> sort,
                                                 int pageSize, int pageNumber,
                                                 out int count)
        {
            IEnumerable<T> result;

            var contextSource = new EntityFrameworkContext();

            EntitiesContext context = contextSource.Context;

            ObjectQuery<T> originalQuery = CreateQuery<T>(context);
            IQueryable<T> query = originalQuery;

            if (filter != null)
                query = query.Where(filter);

            if (sort != null)
                query = query.OrderBy(sort);

            if (pageSize > 0)
            {
                int pageIndex = pageNumber > 0 ? pageNumber - 1 : 0;
                query = query.Skip(pageIndex).Take(pageSize);

                count = query.Count();
            }
            else 
                count = -1;


            result = ExecuteQuery(context, query);

            //if no paging total count is count of the entire result set
            if (count == -1) count = result.Count();

            return result;
        }

        protected internal event Action<ObjectContext, IEnumerable> EntitiesFound;

        protected void OnEntitiesFound<T>(ObjectContext context, params T[] entities)
        {
            if (EntitiesFound != null && entities != null && entities.Length > 0)
            {
                EntitiesFound(context, entities);
            }
        }

        //Allowing room for system-specific-requirement extensibility
        protected Action<IEnumerable> ItemsFound;

        protected IEnumerable<T> ExecuteQuery<T>(ObjectContext context, IQueryable<T> query)
        {
            IEnumerable<T> result = null;

            if (query is ObjectQuery)
            {
                var objectQuery = (ObjectQuery<T>) query;

                objectQuery.EnablePlanCaching = false;
                objectQuery.MergeOption = MergeOption.PreserveChanges;

                result = new List<T>(objectQuery);

                if (ItemsFound != null)
                    ItemsFound(result);

                return result;
            }

            return result;
        }

        internal static RelationshipManager GetRelationshipManager(object entity)
        {
            var entityWithRelationships = entity as IEntityWithRelationships;
            if (entityWithRelationships != null)
            {
                return entityWithRelationships.RelationshipManager;
            }

            return null;
        }


        protected ObjectQuery<T> CreateQuery<T>(ObjectContext context)
        {
            ObjectQuery<T> query = context.CreateQuery<T>(GetEntitySetName(typeof (T)));
            query = this.AggregateEntities(query);
            return query;
        }

        protected virtual ObjectQuery<T> AggregateEntities<T>(ObjectQuery<T> query)
        {
            return query;
        }

        private static string GetEntitySetName(Type entityType)
        {
            return string.Format("{0}Set", entityType.Name);
        }
    }

    public class EntityFrameworkContext
    {
        private const string CtxKey = "ctx";

        private bool contextInitialized
        {
            get { return HttpContext.Current.Items[CtxKey] != null;  }
        }

        public EntitiesContext Context
        {
            get
            {
                if (contextInitialized == false)
                {
                    HttpContext.Current.Items[CtxKey] = new EntitiesContext(ConfigurationManager.ConnectionStrings["CoonectionStringName"].ToString());
                }

                return (EntitiesContext)HttpContext.Current.Items[CtxKey];
            }
        }

        public void TrulyDispose()
        {
            if (contextInitialized)
            {
                Context.Dispose();
                HttpContext.Current.Items[CtxKey] = null;
            }
        }
    }

    internal static class EntityFrameworkExtensions
    {
        internal static ObjectQuery<T> Include<T>(this ObjectQuery<T> query,
                                                  Expression<Func<T, object>> propertyToInclude)
        {
            string include = string.Join(".", propertyToInclude.Body.ToString().Split('.').Skip(1).ToArray());

            const string collectionsLinqProxy = ".First()";
            include = include.Replace(collectionsLinqProxy, "");

            return query.Include(include);
        }

        internal static string After(this string original, string search)
        {
            if (string.IsNullOrEmpty(original))
                return string.Empty;

            int index = original.IndexOf(search);
            return original.Substring(index + search.Length);
        }
    }
}

In Conery's MVC Storefront he created another layer called the "Service" layer which received IQueryable results from the respository and was responsible for applying various filters.

In all cases nobody should be interacting with the repository directly except the services layer.

Most flexible thing is to let Services interact with Repository whatever way they want, same as in above code (yet through one single point -as in example also- to write DRY code and find a place for optimization).
However, the more right way in terms of common DDD patterns is to use the "Specification" pattern, where you encapsulate all your filters, etc in Variables (Class Members, in LINQ typically of delegate types). LINQ can take big optimization benefit out of this when you combine it with "Compiled queries". If you google the {Specification Pattern} and {LINQ Compiled Queries} you'll get closer to what I mean here.

Answer 3

Exposing an IQueryable is a very viable solution and this is how most of the Repository implementations out there doing right now. (Including SharpArchitecture and FubuMVC contrib, as well.)

This is where you are wrong:

However, if you aren't using IQueryable then methods like GetAll() aren't really practical since lazy evaluation won't be taking place down the line. I don't want to return 10,000 records only to use 10 of them later.

This is not realy true. Your example is correct and you should rename GetAll() to a more informative name.

It DOESN'T return all of the items if you call it. That is what IQueryable is for. The concept is called "deferred loading", as it only loads the data (and makes database requests) if you enumerate it.

So, let's say I have a method like this:

IQueryable<T> Retrieve() { ... }

Then, I can call it like this:

Repository.Retrieve<Customer>().Single(c => c.ID == myID);

This ONLY retrieves one row from the database.

And this:

Repository.Retrieve<Customer>().Where(c => c.FirstName == "Joe").OrderBy(c => c.LastName);

This also generates a corresponding query and is only executed when you enumerate it. (It generates an expression tree from the query, and then the query provider should translate that into an appropriate query against the data source.)

You can read more about it in this MSDN article.

Answer 4

I ended up creating two sets of methods, ones that return IEnumerable (in your case IQueryable), and ones that return Collection (pull the content before sending it out of the repository.)

This allows me to do both build ad hoc queries in Services outside the repository as well as use Repository methods directly returning side effect resistant Collections. In other words, joining two Repository Entities together results in one select query, instead of one select query for each entity found.

I imagine you could set your protection level to keep truly bad things from happening.