As per the traditional approach or theory wise, ViewModel should be part of User Interface(UI) layer. At least the name says so.
But when you get down to implementing it yourself with Entity Framework, MVC, Repository etc, then you realise something else.
Someone has to map Entity Models with ViewModels(DTO mentioned in the end). Should this be done in A) the UI layer (by the Controller), or in B) the Service layer?
I go with Option B. Option A is a no-no because of the simple fact that several entity models combine together to form a ViewModel. We may not pass unnecessary data to UI layer, whereas in option B, the service can play with data and pass only the required/minimum to the UI layer after mapping (to the ViewModel).
But, let us assume we go with Option A, we put ViewModel in the UI layer(and entity model in Service layer).
If the Service layer needs to map to the ViewModel, then the Service layer need to access ViewModel in UI layer. Which library/project? The Viewmodel should be in a separate project in the UI layer, and this project needs to be referenced by Service Layer. If the ViewModel is not in a separate project(.dll), then there is circular reference, so no go. It looks awkward to have Service layer accessing UI layer but still we could cope with it.
But what if there is another UI app using this service? What if there is a mobile app? How different can the ViewModel be? Should the Service access the same view model project? or will all UI projects compete?
After these considerations my answer would be to put the Viewmodel project in Service Layer. Every UI layer has to access the Service layer anyways! And there could be a lot of similar ViewModels that they all could use (hence mapping becomes easier for service layer). Mappings are done through linq these days, which is another plus.
Lastly there is this discussion about DTO. And also about data annotation in ViewModels. ViewModels with data annotations cannot reside in service layer. So then DTO will be an exact copy of ViewModel with a one on one mapping between the two(say with AutoMapper). Again DTO still has the logic needed for the UI(or multiple applications) and resides in Service Layer. And the UI layer ViewModel is just to copy the data from DTO, with some 'behaviour'(eg: attribute) added to it.
[Now this discussion is about to take an interesting turn read on...:I]
And don't think data-annotation attributes are just for UI. If you limit the validation using System.ComponentModel.DataAnnotations.dll then the same ViewModel can also be used for front-end & backend validation(thus removing UI-residing-ViewModel-copy-of-DTO). Moreover attributes can also be used in entity models. Eg: using .tt, Entity Framework data models can be autogenerated with validation attributes to do some DB validations like max-length before sending to the back end. Another advantage is that if backend validation changes in DB then .tt (reads DB specifics and create the attribute for entity class) will automatically pick that up. This can force UI validation unit tests to fail as well, which is a big plus(so we can correct it and inform all UIs/consumers instead of accidentally forgetting and failing). Yes, the discussion is moving towards a good framework design. As you can see it is all related: tier-wise validation, unit test strategy, caching strategy, etc.
Although not directly related to the question. 'ViewModel Façade' (viewmodel inside another viewmodel) & 'command' mentioned in this must watch channel 9 link is also worth exploring(@11:48 it starts)