Asynchronously Lazy-Loading Navigation Properties of detached Self-Tracking Entities through a WCF service?

Question

I have a WCF client which passes Self-Tracking Entities to a WPF application built with MVVM. The application itself has a dynamic interface. Users can select which objects they want visible in their Work area depending on what role they are in or what task they are doing.

My self-tracking entities have quite a few Navigation Properties, and a lot of them are not needed. Since some of these objects can be quite large, I'd like to only load these properties on request.

My application looks like this:

[WCF] <---> [ClientSide Repository] <---> [ViewModel] <---> [View]

My Models are Self-Tracking Entities. The Client-Side Repository hooks up a LazyLoad method (if needed) before returning the Model to the ViewModel that requested it. All WCF Service calls are asyncronous, which means the LazyLoad methods are also asyncronous.

The actual implementation of the LazyLoad is giving me some trouble. Here are the options I have come up with.

EDIT - I removed the code samples to try and make this easier to read and understand. See previous version of question if you want to see it

Option A

Asynchronously LazyLoad the Model's properties from the WCF server in the Getter

Good: Loading data on demand is extremely simple. The binding in the XAML loads the data, so if the control is on the screen the data loads asynchronsly and notifies the UI when it's there. If not, nothing loads. For example, <ItemsControl ItemsSource="{Binding CurrentConsumer.ConsumerDocuments}" /> will load the data, however if the Documents section of the interface is not there then nothing gets loaded.

Bad: Cannot use this property in any other code before it has been initiated because it will return an empty list. For example, the following call will always return false if documents have not been loaded.

public bool HasDocuments 
{ 
    get { return ConsumerDocuments.Count > 0; }
}

OPTION B

Manually make a call to load data when needed

Good: Simple to implement - Just add LoadConsumerDocumentsSync() and LoadConsumerDocumentsAsync() methods

Bad: Must remember to load the data before trying to access it, including when its used in Bindings. This might seem simple, but it can get out of hand quickly. For example, each ConsumerDocument has a UserCreated and UserLastModified. There is a DataTemplate that defines the UserModel with a ToolTip displaying additional User data such as extension, email, teams, roles, etc. So in my ViewModel that displays documents I would have to call LoadDocuments, then loop through them and call LoadConsumerModified and LoadConsumerCreated. It could keep going too... after that I'd have to LoadUserGroups and LoadUserSupervisor. It also runs the risk of circular loops where something like a User has a Groups[] property, and a Group has a Users[] property

OPTION C

My favorite option so far... create two ways to access the property. One Sync and one Async. Bindings would be done to the Async property and any code would use the Sync property.

Good: Data is loaded asynchronously as needed - Exactly what I want. There isn't that much extra coding either since all I would need to do is modify the T4 template to generate these extra properties/methods.

Bad: Having two ways to access the same data seems inefficient and confusing. You'd need to remember when you should use Consumer.ConsumerDocumentsAsync instead of Consumer.ConsumerDocumentsSync. There is also the chance that the WCF Service call gets run multiple times, and this requires an extra IsLoaded property for every navigational property, such as IsConsumerDocumentsLoaded.

OPTION D

Skip the Asyncronous loading and just load everything synchronously in the setters.

Good: Very simple, no extra work needed

Bad: Would lock the UI when data loads. Don't want this.

OPTION E

Have someone on SO tell me that there is another way to do this and point me to code samples :)

Other Notes

Some of the NavigationProperties will be loaded on the WCF server prior to returning the object to the client, however others are too expensive to do that with.

With the exception of manually calling the Load events in Option C, these can all be done through the T4 template so there is very little coding for me to do. All I have to do is hook up the LazyLoad event in the client-side repository and point it to the right service calls.

Corrasion

I guess it is related to your other questions. Why do you use STEs here if you do not plan to use these entities for updates? – Earmark

@Ladislav It is, thanks for answering my other questions. We wanted change tracking on the object's base properties, and an argument was made for using STEs over creating our own change-tracking system. The STEs will be used for updates, but most navigational properties will get updated separately. For example, a Consumer can be a Person or a Business. These entities will update the base Consumer object when they update, but something like ConsumerDocuments would be updated separately. – Corrasion

Answer 1

The solution I came up with was to modify the T4 template for the self-tracking entities to make the changes shown below. The actual implementation has been omitted to make this easier to read, but the property/method names should make it clear what everything does.

Old T4 Generated Navigation Properties

[DataMember]
public MyClass MyProperty { get; set;}

private MyClass _myProperty;

New T4 Generated Navigation Properties

[DataMember]
internal MyClass MyProperty {get; set;}
public MyClass MyPropertySync {get; set;}
public MyClass MyPropertyAsync {get; set;}

private MyClass _myProperty;
private bool _isMyPropertyLoaded;

private async void LoadMyPropertyAsync();
private async Task<MyClass> GetMyPropertyAsync();
private MyClass GetMyPropertySync();

I created three copies of the property, which point to the same private property. The internal copy is for EF. I could probably get rid of it, but its easiest to just leave it in since EF expects a property by that name and its easier to leave it than to fix up EF to use a new property name. It is internal since I don't want anything outside of the class namespace to use it.

The other two copies of the property run the exact same way once the value has been loaded, however they load the property differently.

The Async version runs LoadMyPropertyAsync(), which simply runs GetMyPropertyAsync(). I needed two methods for this because I cannot put the async modifier on a getter, and I need to return a void if calling from a non-async method.

The Sync version runs GetMyPropertySync() which in turn runs GetMyPropertyAsync() synchronously

Since this is all T4-generated, I don't need to do a thing except hook up the async lazy load delegate when the entity is obtained from the WCF service.

My bindings point to the Async version of the property and any other code points to the Sync version of the property and both work correctly without any extra coding.

<ItemsControl ItemsSource="{Binding CurrentConsumer.DocumentsAsync}" />

CurrentConsumer.DocumentsSync.Clear();

Corrasion

Answer 2

Gave it some thought, first of all I have to say that you must provide a clear to reader solution to this problem, DependecyProperties being loaded async when you bind to User.Documents property can be ok, but its pretty close to a side effect based solution. If we say that such behaviour in View is ok, we must keep our rest code very clear about it intentions, so we can see how are we trying to access data - async or sync via some verbose naming of something (method,classname,smth else).

So I think we could use a solution that is close to old .AsSynchronized() approach, create a decorator class, and provide each property a private/protected AsyncLoad & SyncLoad method, and a decorator class would be Sync or Async version of each lazyloadable class, whatever is more appropraite.

When you decorate your class with Sync decorator it wraps each lazyloadable class inside with a Sync decorator too so you will be able to use SynchUser(User).Documents.Count on sync class version with no probs cause it will be smth like SynchUser(user).SyncDocuments(Documents).Count behind in overloaded version of Documents property and would call sync getter function.

Since both sync and async versions will be operating on same object this approach wont lead to modifying some non referenced anywhere else object if you want to modify any property.

Your task may sound as one that can be solved in some magic "beautiful & simple" way but I dont think it can, or that it wont be any more simple than this one.

If this doesn't work im still 100% sure you need a clear way to differntiate in code whether a sync or async version of class is used or you will have a very hard to maintain code base.

Herndon

I'm fairly sure you're right, that this can't be accomplished in some "beautiful and simple" way. I am currently working towards something similar right now... I am creating two copies of the public property: one with the Sync suffix and one with the Async suffix. Both will operate on the same Private property. It should be fairly easy to figure out which to call or to bind to based on what you are doing – Corrasion

this way you will have an overloaded interface,with most members X2 while in reality your code rarely needs async methods, and View rarely will need sync ones. I am also thinking that async ones will be used almost exclusively in View so it would be benefitial to leave sync version default. I think having a version of class that is either sync or async instead of having 2x methods everywhere will make a better design. Also Sync and Async can be overlooked and misseen,if there is some rare used way to turn class into async mode(just for view) use you would be able to prevent a lot of errors. – Herndon

Would you have an example of what sort of code you would use? I think I understand what you're trying to say, but I have no idea how I'd implement it. – Corrasion

Answer 3

Option A should be the solution.

Create one property named LoadingStatus indicating data is loaded or loading not yet loaded. Load data asynchronously and set the LoadingStatus property accordingly.

Check the loading status in each property and if data not loaded then call function to load data and viceversa.

Coward

Answer 4

Could the Binding.IsAsync library property be helpful here?

Edit: expanding a little.. Have a lazy loaded synchronous property that will call the WCF service on first use. Then the async binding will keep the UI from blocking.

Ezekiel

Nope, that simply makes the binding Async. Something still needs to call the WCF server to get the data asynchronously – Corrasion

Answer 5

While this question was asked a while ago, it is near the top of the async-await keyword list and I think would be answered quite differently in .net 4.5.

I believe this would be a perfect use case for the AsyncLazy<T> type described on several sites:

http://blogs.msdn.com/b/pfxteam/archive/2011/01/15/10116210.aspx http://blog.stephencleary.com/2012/08/asynchronous-lazy-initialization.html http://blog.stephencleary.com/2013/01/async-oop-3-properties.html

Selenaselenate

Answer 6

I have two thoughts in my head.

1) Implement an IQueryable<> response on the WCF Service. And follow right down to the DB with an IQueryable<> pattern.

2) In the client repository set the getter on the ConsumerDocuments property to fetch the data.

private IEnumerable<ConsumerDocuments> _consumerDocuments;

public IEnumerable<ConsumerDocuments> ConsumerDocuments
{
    get
    {
        return _consumerDocuments ?? (_consumerDocuments = GetConsumerDocuments() );
    }
}

Gamophyllous

My problem with that is the GetConsumerDocuments() is Async, so something calling ConsumerDocuments.Count is going to get a 0 every time unless the Async method has been run before. And I don't want to make it Sync because that defeats the whole point of doing in Async in the first place and will lock up the UI anytime a binding occurs to a property that hasn't been initialized yet. – Corrasion

in option 2) above, calling .Count() would call the server for the data first. If you want the count, but not the actual data. You could have that as an additional property in the Consumer i.e. int ConsumerDocumentsCount – Gamophyllous

.Count() was simply an example. The same thing would happen if any code called this property before it was initialized. You are basically saying to do the same thing that I have listed in Option A – Corrasion

would option 2) not cover this? as soon as Consumer.ConsumerDocuments is accessed it is then pulled from the server. – Gamophyllous

The call to get the documents from the server is Asynchronous. So it creates a blank collection, then makes a call to the server to get documents in an async operation. Once the call finishes, the Docs get added to the Collection and INotifyPropertyChanged gets called and UI updates. Meanwhile, while documents are getting requested from the server and downloaded the user can continue with whatever they were doing without their UI locking up – Corrasion

Answer 7

The way I see it, the ViewModel needs to be aware if there's data available or not. You can hide or disable UI elements that are meaningless without data while the data is being fetched, then show them when the data arrives.

You detect that you need to load in some data, so you set the UI to "waiting" mode, kick off the async fetch, then when the data comes in take it out of waiting mode. Perhaps by having the ViewModel subscribe to a "LoadCompleted" event on the object it's interested in.

(edit) You can avoid excessive loads or circular dependencies by keeping track of the state of each model object: Unloaded/Loading/Loaded.

Thistledown

That sounds like my Option B where you call the Load events manually. Ideally, I do not want to do this because then I have to concern myself with loading every single bit of data that my Views will use. I would prefer to simply have my View call a property on my Model, and have it get loaded asynchronously so it does not lock up the UI – Corrasion

Well you can't have both the requirement to call a property on your ViewModel and to not block on the UI thread. There's always the ObjectDataProvider for async data binding, but that's going to need to have an async fetch method for every field. Since I don't think you want to make a separate async call for every little field, you'll have to do a bit of management yourself. What I would do is on each model, have IsLoaded, IsLoading, LoadAsync and LoadCompleted. The ViewModel can use these to get a value or ensure a load is underway, give a fallback value and subscribe to the load completion. – Thistledown

The way I am leaning towards is a single private property, two public properties (one Async for XAML Bindings, one Sync for synchronous code that depends on a value being valid), and an IsLoaded property to determine if data is currently valid or not. I am specifically looking at STE's NavigationalProperties, which are not loaded on the WCF service at the time the object is created. Some NavigationalProperties will get loaded on the WCF server, others I want to be lazy-loaded as needed. I am using C#5.0's await/async keywords for Asynchronous calls. – Corrasion

Well I usually like to avoid having two different modes of getting data. Since the data binding is happening against synchronous properties anyway, I like the "async loading, sync access" model. – Thistledown

Answer 8

Here is an option E for you.

Asynchronously load data. Have the initial fetch queue things up in a a background thread that fills out the full objects slowly. And make any methods that require data be loaded behind the scenes be blocking on the load finishing. (Blocking and have them notify the background thread that the data that they need is high priority, get them next, so you can unblock ASAP.)

This gives you a UI that is immediately responsive when it can be, the ability to write your code and not think about what has been loaded, and it will mostly just work. The one gotcha is that occasionally you'll make a blocking call while data is loading, however hopefully it won't do that too often. If you do, then in the worst case you degrade to something like option C where you have both a blocking fetch of data, and the ability to poll to see if it is there. However most of the time you wouldn't have to worry too much about it.

Disclaimer: I personally don't use Windows, and spend most of my time working on back ends far from UIs. If you like the idea, feel free to try it. But I haven't actually followed this strategy for anything more complicated than some behind the scenes AJAX calls in a dynamic web page.

Adama

That might result in a lot of extra network traffic because objects that aren't needed are getting loaded – Corrasion

@Corrasion - Yes, it will result in extra network traffic. Without knowing your application I can't say whether it will be significant. However I can say that it is reasonably rare to have bandwidth be the major bottleneck in an intranet situation. In an internet situation, take a look at Google for what is possible. – Adama