ASP.NET MVC3 and Entity Framework Code first architecture

Question

My previous question made me think again about layers, repository, dependency injection and architectural stuff like this.

My architecture now looks like this:
I am using EF code first, so I just made POCO classes, and context. That creates db and model.
Level higher are business layer classes (Providers). I am using different provider for each domain... like MemberProvider, RoleProvider, TaskProvider etc. and I am making new instance of my DbContext in each of these providers.
Then I instantiate these providers in my controllers, get data and send them to Views.

My initial architecture included repository, which I got rid of because I was told that it just adds complexity, so why I don't just use EF only. I wanted to did that.. working with EF directly from controllers, but I have to write tests and it was a bit complicate with real database. I had to fake - mock data somehow. So I made an interface for each provider and made fake providers with hardcoded data in lists. And with this I got back to something, where I am not sure how to proceed correctly.

These things starts to be overcomplicated too quickly... many approaches and "pattterns"... it creates just too much noise and useless code.

Is there any SIMPLE and testable architecture for creating and ASP.NET MVC3 application with Entity Framework?

Purvis

You said repositories added 'complexity' to your application, but I would say that they're an initial 'overhead' that makes testing easier. Mocking a few repositories is easier than mocking an entire data context. – Lehet

Yes, but I don't want that initial overhead in my current case. I want to progress with application quickly. I already lost too much time without any real progress. Adding repositories brings things like IoC, DI etc.. and I will have to write zillions of tests before I get to first View. I know that this might be right solution, but I am not looking for "the correct one". I am looking for simple (while still testable) solution. – Purvis

Answer 1

If you want to use TDD (or any other testing approach with high test coverage) and EF together you must write integration or end-to-end tests. The problem here is that any approach with mocking either context or repository just creates test which can test your upper layer logic (which uses those mocks) but not your application.

Simple example:

Let's define generic repository:

public interface IGenericRepository<TEntity> 
{
    IQueryable<TEntity> GetQuery();
    ...
}

And lets write some business method:

public IEnumerable<MyEntity> DoSomethingImportant()
{
    var data = MyEntityRepo.GetQuery().Select((e, i) => e);
    ...
}

Now if you mock the repository you will use Linq-To-Objects and you will have a green test but if you run the application with Linq-To-Entities you will get an exception because select overload with indexes is not supported in L2E.

This was simple example but same can happen with using methods in queries and other common mistakes. Moreover this also affects methods like Add, Update, Delete usually exposed on repository. If you don't write a mock which will exactly simulate behavior of EF context and referential integrity you will not test your implementation.

Another part of story are problems with Lazy loading which can also hardly be detected with unit tests against mocks.

Because of that you should also introduce integration or end-to-end tests which will work against real database using real EF context ane L2E. Btw. using end-to-end tests is required to use TDD correctly. For writing end-to-end tests in ASP.NET MVC you can WatiN and possibly also SpecFlow for BDD but this will really add a lot of work but you will have your application really tested. If you want to read more about TDD I recommend this book (the only disadvantage is that examples are in Java).

Integration tests make sense if you don't use generic repository and you hide your queries in some class which will not expose IQueryable but returns directly data.

Example:

public interface IMyEntityRepository
{
    MyEntity GetById(int id);
    MyEntity GetByName(string name); 
}

Now you can just write integration test to test implementation of this repository because queries are hidden in this class and not exposed to upper layers. But this type of repository is somehow considered as old implementation used with stored procedures. You will lose a lot of ORM features with this implementation or you will have to do a lot of additional work - for example add specification pattern to be able to define query in upper layer.

In ASP.NET MVC you can partially replace end-to-end tests with integration tests on controller level.

Edit based on comment:

I don't say that you need unit tests, integration tests and end-to-end tests. I say that making tested applications require much more effort. The amount and types of needed tests is dependent on the complexity of your application, expected future of the application, your skills and skills of other team members.

Small straighforward projects can be created without tests at all (ok, it is not a good idea but we all did it and at the end it worked) but once a project passes some treshold you can find that introducing new features or maintaining the project is very hard because you are never sure if it breaks something which already worked - that is called regression. The best defence against regression is good set of automated tests.

• Unit tests help you to test method. Such tests should ideally cover all execution paths in the method. These tests should be very short and easy to write - to complicated part can be to set up dependencies (mocks, faktes, stubs).
• Integration tests help you to test functionality accross multiple layers and usually accross multiple processes (application, database). You don't need to have them for everything, it is more about experience to select where they are helpful.
• End-to-end tests are something like validation of use case / user story / feature. They should cover whole flow of the requirement.

It is not needed to test a feture multiple times - if you know that the feature is tested in end-to-end test you don't need to write integration test for the same code. Also if you know that method has only single execution path which is covered by integration test you don't need to write unit test for it. This works much better with TDD approach where you start with a big test (end-to-end or integration) and go deeper to unit tests.

Depending on your developement approach you don't have to start with multiple types of test from beginning but you can introduce them later as your application will become more complex. The exception is TDD/BDD where you should start to use at least end-to-end and unit tests before you even write single line of other code.

So you are asking the wrong question. The question is not what is simpler? The question is what will help you at the end and what complexity fits your application? If you want to have easily unit tested application and business logic you should wrap EF code to some other classes which can be mocked. But in the same time you must introduce other type of tests to ensure that EF code works.

I can't say you what approach will fit your environment / project / team / etc. But I can explain example from my past project:

I worked on the project for about 5-6 months with two collegues. The project was based on ASP.NET MVC 2 + jQuery + EFv4 and it was developed in incremental and iterative way. It had a lot of complicated business logic and a lot of complicated database queries. We started with generic repositories and high code coverage with unit tests + integration tests to validate mapping (simple tests for inserting, deleting, updating and selecting entity). After few months we found that our approach doesn't work. We had more then 1.200 unit tests, code coverage about 60% (that is not very good) and a lot of regression problems. Changing anything in EF model could introduce unexpected problems in parts which were not touched for several weeks. We found that we are missing integration tests or end-to-end tests for our application logic. The same conclusion was made on a parallel team worked on another project and using integration tests was considered as recommendation for new projects.

Travail

Hmm.. so if I understood correctly, you are saying that using mocks is for unit testing of business logic and I need to do integration tests with real ef context as well as end-to-end tests (I understand this as functional/user tests.. with tools like Watin). But I don't get the point about architecture. I am glad that you gave me hints about what the problems are, but I am not experienced in this field so I do not know what is a better solution. And that's what I am looking for here. And I am talking about better in "easier" or "simple" meaning. – Purvis

Thank you. I highly appreciate your reponses and explanations. I believe I am using the "wrap ef code to other class" (my Provider classes) right now. And just to add some context into my questions: I am creating simple application built around task (in project context) management for users (+ expert system, which doesn't change the architecture anyway becuase it's just consuming data and provides simple output). It's my own project (nobody else working on it) and I don't think it will have any great future. – Purvis

@Ladislav: Is it possible that static code analysis tools can catch issues like the one you described (unsupported Linq methods for Linq to Entities)? If so, then you could eliminate a class of errors without having to write unit tests for them, and get more confidence that your mocks in the tests that you write will "really work". It probably couldn't solve referential integrity issues, but as you said, those can be taken care of with integration tests (rather than E2E). – Jadda

Testability is exactly the reason I do not expose IQueryable from repositories. I do get larger repositories but with methods that have clear responsibilities and are whole lot easier to mock. – Pearle

Answer 2

Does using repository pattern add complexity? In your scenario I don't think so. It makes TDD easier and your code more manageable. Try to use a Generic repository pattern for more separation and cleaner code.

If you want to find out more about TDD and design patterns in Entity Framework, take a look at: http://msdn.microsoft.com/en-us/ff714955.aspx

However it seems like you're looking for an approach to mock test Entity Framework. One solution would be using a virtual seed method to generate data on database initialization. Take a look at Seed section at: http://blogs.msdn.com/b/adonet/archive/2010/09/02/ef-feature-ctp4-dbcontext-and-databases.aspx

Also you can use some mocking frameworks. The most famous ones I know are:

• Rhino Mocks
• Moq
• Typemock (Commercial)

To see a more complete list of .NET mocking frameworks, check out: https://stackoverflow.com/questions/37359/what-c-mocking-framework-to-use

Another approach would be to use an in-memory database provider like SQLite. Study more at Is there an in-memory provider for Entity Framework?

Finally, here are some good links about unit testing Entity Framework (Some links refer to Entity Framework 4.0. But you'll get the idea.):

http://social.msdn.microsoft.com/Forums/en/adodotnetentityframework/thread/678b5871-bec5-4640-a024-71bd4d5c77ff

http://mosesofegypt.net/post/Introducing-Entity-Framework-Unit-Testing-with-TypeMock-Isolator.aspx

What is the way to go to fake my database layer in a unit test?

Galle

Thanks for your input, some interesting links are there. But my question actually isn't that much about testing and mocks. It's more about looking for simple, quick an easy architecture without overhead. Something that you can use to test and develop applications quickly and easily without preparing XYZ lines of codes to see, if your method really does return a string value. {Sry for a little bit of sarcasm.} – Purvis

@dampe: Well, instead of writing interfaces and go with manually mocking data, I suggested some additional solutions which can do a lot of work for you. Once again, I'd go with a generic repository pattern in these cases and never felt it adds complexity to my solution. Hope it helps. – Galle

Regarding the suggestion about a generic repository, see this tutorial: asp.net/entity-framework/tutorials/… – Irving

Answer 3

What I do is I use a simple ISession and EFSession object, witch are easy to mock in my controller, easy to access with Linq and strongly typed. Inject with DI using Ninject.

public interface ISession : IDisposable
    {
        void CommitChanges();
        void Delete<T>(Expression<Func<T, bool>> expression) where T : class, new();
        void Delete<T>(T item) where T : class, new();
        void DeleteAll<T>() where T : class, new();
        T Single<T>(Expression<Func<T, bool>> expression) where T : class, new();
        IQueryable<T> All<T>() where T : class, new();
        void Add<T>(T item) where T : class, new();
        void Add<T>(IEnumerable<T> items) where T : class, new();
        void Update<T>(T item) where T : class, new();
    }

public class EFSession : ISession
    {
        DbContext _context;

        public EFSession(DbContext context)
        {
            _context = context;
        }


        public void CommitChanges()
        {
            _context.SaveChanges();
        }

        public void Delete<T>(System.Linq.Expressions.Expression<Func<T, bool>> expression) where T : class, new()
        {

            var query = All<T>().Where(expression);
            foreach (var item in query)
            {
                Delete(item);
            }
        }

        public void Delete<T>(T item) where T : class, new()
        {
            _context.Set<T>().Remove(item);
        }

        public void DeleteAll<T>() where T : class, new()
        {
            var query = All<T>();
            foreach (var item in query)
            {
                Delete(item);
            }
        }

        public void Dispose()
        {
            _context.Dispose();
        }

        public T Single<T>(System.Linq.Expressions.Expression<Func<T, bool>> expression) where T : class, new()
        {
            return All<T>().FirstOrDefault(expression);
        }

        public IQueryable<T> All<T>() where T : class, new()
        {
            return _context.Set<T>().AsQueryable<T>();
        }

        public void Add<T>(T item) where T : class, new()
        {
            _context.Set<T>().Add(item);
        }

        public void Add<T>(IEnumerable<T> items) where T : class, new()
        {
            foreach (var item in items)
            {
                Add(item);
            }
        }

        /// <summary>
        /// Do not use this since we use EF4, just call CommitChanges() it does not do anything
        /// </summary>
        /// <typeparam name="T"></typeparam>
        /// <param name="item"></param>
        public void Update<T>(T item) where T : class, new()
        {
            //nothing needed here
        }

If I want to switch from EF4 to let's say MongoDB, I only have to make a MongoSession that implement ISession...

Anta

Thanks. I believe that I am doing something similar... except that generic and Ninject part :) – Purvis

Answer 4

I was having the same problem deciding on the general design of my MVC application. This CodePlex project by Shiju Varghese was alot of help. It is done in ASP.net MVC3, EF CodeFirst and also utilizes a service layer and a repository layer as well. Dependency injection is done using Unity. It is simple and very easy to follow. It is also backed by some 4 very nice blog posts. Its worth checking out. And, don't give up on the repository..yet.

Scan

Thanks, I took a look at that solution code and it does almost exaclty what I don't want... all that repositories, IoC, factories etc. This is not what I imagine when someone says "simple architecture" :) – Purvis

The simplest design I can suggest (although not recommend) is creating the EF Context objects directly from your controllers, but as indicated in your question you already tried that and are having problems with it already.. – Scan