Serializing Entity Framework problems

Question

Like several other people, I'm having problems serializing Entity Framework objects, so that I can send the data over AJAX in a JSON format.

I've got the following server-side method, which I'm attempting to call using AJAX through jQuery

[WebMethod]
public static IEnumerable<Message> GetAllMessages(int officerId)
{

        SIBSv2Entities db = new SIBSv2Entities();

        return  (from m in db.MessageRecipients
                        where m.OfficerId == officerId
                        select m.Message).AsEnumerable<Message>();
}

Calling this via AJAX results in this error:

A circular reference was detected while serializing an object of type \u0027System.Data.Metadata.Edm.AssociationType

Which is because of the way the Entity Framework creates circular references to keep all the objects related and accessible server side.

I came across the following code from (http://hellowebapps.com/2010-09-26/producing-json-from-entity-framework-4-0-generated-classes/) which claims to get around this problem by capping the maximum depth for references. I've added the code below, because I had to tweak it slightly to get it work (All angled brackets are missing from the code on the website)

using System.Web.Script.Serialization;
using System.Collections.Generic;
using System.Collections;
using System.Linq;
using System;


public class EFObjectConverter : JavaScriptConverter
{
  private int _currentDepth = 1;
  private readonly int _maxDepth = 2;

  private readonly List<int> _processedObjects = new List<int>();

  private readonly Type[] _builtInTypes = new[]{
    typeof(bool),
    typeof(byte),
    typeof(sbyte),
    typeof(char),
    typeof(decimal),
    typeof(double),
    typeof(float),
    typeof(int),
    typeof(uint),
    typeof(long),
    typeof(ulong),
    typeof(short),
    typeof(ushort),
    typeof(string),
    typeof(DateTime),
    typeof(Guid)
  };

  public EFObjectConverter( int maxDepth = 2,
                            EFObjectConverter parent = null)
  {
    _maxDepth = maxDepth;
    if (parent != null)
    {
      _currentDepth += parent._currentDepth;
    }
  }

  public override object Deserialize( IDictionary<string,object> dictionary, Type type, JavaScriptSerializer serializer)
  {
    return null;
  }     

  public override IDictionary<string,object> Serialize(object obj, JavaScriptSerializer serializer)
  {
    _processedObjects.Add(obj.GetHashCode());
    Type type = obj.GetType();
    var properties = from p in type.GetProperties()
                      where p.CanWrite &&
                            p.CanWrite &&
                            _builtInTypes.Contains(p.PropertyType)
                      select p;
    var result = properties.ToDictionary(
                  property => property.Name,
                  property => (Object)(property.GetValue(obj, null)
                              == null
                              ? ""
                              :  property.GetValue(obj, null).ToString().Trim())
                  );
    if (_maxDepth >= _currentDepth)
    {
      var complexProperties = from p in type.GetProperties()
                                where p.CanWrite &&
                                      p.CanRead &&
                                      !_builtInTypes.Contains(p.PropertyType) &&
                                      !_processedObjects.Contains(p.GetValue(obj, null)
                                        == null
                                        ? 0
                                        : p.GetValue(obj, null).GetHashCode())
                              select p;

      foreach (var property in complexProperties)
      {
        var js = new JavaScriptSerializer();

          js.RegisterConverters(new List<JavaScriptConverter> { new EFObjectConverter(_maxDepth - _currentDepth, this) });

        result.Add(property.Name, js.Serialize(property.GetValue(obj, null)));
      }
    }

    return result;
  }

  public override IEnumerable<System.Type> SupportedTypes
  {
    get
    {
      return GetType().Assembly.GetTypes();
    }
  }

}

However even when using that code, in the following way:

    var js = new System.Web.Script.Serialization.JavaScriptSerializer();
    js.RegisterConverters(new List<System.Web.Script.Serialization.JavaScriptConverter> { new EFObjectConverter(2) });
    return js.Serialize(messages);

I'm still seeing the A circular reference was detected... exception being thrown!

Sperm

possible duplicate of Serialize Entity Framework objects into JSON – Formenti

That question goes with anonymous types as a solution, whereas I'm taking a different approach in the solution I'm trying to get working – Sperm

Is there any reason you aren't happy projecting onto an anonymous type (or a statically defined type). Surely your complex type needs flattening in some way? I'm not sure why the above Converter won't work - are you happy to post the code for the 'Message' class you're trying to serialize? Also - you're still deferring the query's execution in the LINQ query, try calling 'ToList()' after the 'AsEnumerable()' call. – Palmy

David - there are many valid reasons not to want to project on every single entity access. DRY for a start.. consider you have an entity with 40 simple properties and only 1 complex type property which produces a circular reference! – Balsamiferous

Answer 1

I solved these issues with the following classes:

public class EFJavaScriptSerializer : JavaScriptSerializer
  {
    public EFJavaScriptSerializer()
    {
      RegisterConverters(new List<JavaScriptConverter>{new EFJavaScriptConverter()});
    }
  }

and

public class EFJavaScriptConverter : JavaScriptConverter
  {
    private int _currentDepth = 1;
    private readonly int _maxDepth = 1;

    private readonly List<object> _processedObjects = new List<object>();

    private readonly Type[] _builtInTypes = new[]
    {
      typeof(int?),
      typeof(double?),
      typeof(bool?),
      typeof(bool),
      typeof(byte),
      typeof(sbyte),
      typeof(char),
      typeof(decimal),
      typeof(double),
      typeof(float),
      typeof(int),
      typeof(uint),
      typeof(long),
      typeof(ulong),
      typeof(short),
      typeof(ushort),
      typeof(string),
      typeof(DateTime),
      typeof(DateTime?),
      typeof(Guid)
  };
    public EFJavaScriptConverter() : this(1, null) { }

    public EFJavaScriptConverter(int maxDepth = 1, EFJavaScriptConverter parent = null)
    {
      _maxDepth = maxDepth;
      if (parent != null)
      {
        _currentDepth += parent._currentDepth;
      }
    }

    public override object Deserialize(IDictionary<string, object> dictionary, Type type, JavaScriptSerializer serializer)
    {
      return null;
    }

    public override IDictionary<string, object> Serialize(object obj, JavaScriptSerializer serializer)
    {
      _processedObjects.Add(obj.GetHashCode());
      var type = obj.GetType();

      var properties = from p in type.GetProperties()
                       where p.CanRead && p.GetIndexParameters().Count() == 0 &&
                             _builtInTypes.Contains(p.PropertyType)
                       select p;

      var result = properties.ToDictionary(
                    p => p.Name,
                    p => (Object)TryGetStringValue(p, obj));

      if (_maxDepth >= _currentDepth)
      {
        var complexProperties = from p in type.GetProperties()
                                where p.CanRead &&
                                      p.GetIndexParameters().Count() == 0 &&
                                      !_builtInTypes.Contains(p.PropertyType) &&
                                      p.Name != "RelationshipManager" &&
                                      !AllreadyAdded(p, obj)
                                select p;

        foreach (var property in complexProperties)
        {
          var complexValue = TryGetValue(property, obj);

          if(complexValue != null)
          {
            var js = new EFJavaScriptConverter(_maxDepth - _currentDepth, this);

            result.Add(property.Name, js.Serialize(complexValue, new EFJavaScriptSerializer()));
          }
        }
      }

      return result;
    }

    private bool AllreadyAdded(PropertyInfo p, object obj)
    {
      var val = TryGetValue(p, obj);
      return _processedObjects.Contains(val == null ? 0 : val.GetHashCode());
    }

    private static object TryGetValue(PropertyInfo p, object obj)
    {
      var parameters = p.GetIndexParameters();
      if (parameters.Length == 0)
      {
        return p.GetValue(obj, null);
      }
      else
      {
        //cant serialize these
        return null;
      }
    }

    private static object TryGetStringValue(PropertyInfo p, object obj)
    {
      if (p.GetIndexParameters().Length == 0)
      {
        var val = p.GetValue(obj, null);
        return val;
      }
      else
      {
        return string.Empty;
      }
    }

    public override IEnumerable<Type> SupportedTypes
    {
      get
      {
        var types = new List<Type>();

        //ef types
        types.AddRange(Assembly.GetAssembly(typeof(DbContext)).GetTypes());
        //model types
        types.AddRange(Assembly.GetAssembly(typeof(BaseViewModel)).GetTypes());


        return types;

      }
    }
  }

You can now safely make a call like new EFJavaScriptSerializer().Serialize(obj)

Update : since version Telerik v1.3+ you can now override the GridActionAttribute.CreateActionResult method and hence you can easily integrate this Serializer into specific controller methods by applying your custom [GridAction] attribute:

[Grid]
public ActionResult _GetOrders(int id)
{ 
   return new GridModel(Service.GetOrders(id));
}

and

public class GridAttribute : GridActionAttribute, IActionFilter
  {    
    /// <summary>
    /// Determines the depth that the serializer will traverse
    /// </summary>
    public int SerializationDepth { get; set; } 

    /// <summary>
    /// Initializes a new instance of the <see cref="GridActionAttribute"/> class.
    /// </summary>
    public GridAttribute()
      : base()
    {
      ActionParameterName = "command";
      SerializationDepth = 1;
    }

    protected override ActionResult CreateActionResult(object model)
    {    
      return new EFJsonResult
      {
       Data = model,
       JsonRequestBehavior = JsonRequestBehavior.AllowGet,
       MaxSerializationDepth = SerializationDepth
      };
    }
}

and finally..

public class EFJsonResult : JsonResult
  {
    const string JsonRequest_GetNotAllowed = "This request has been blocked because sensitive information could be disclosed to third party web sites when this is used in a GET request. To allow GET requests, set JsonRequestBehavior to AllowGet.";

    public EFJsonResult()
    {
      MaxJsonLength = 1024000000;
      RecursionLimit = 10;
      MaxSerializationDepth = 1;
    }

    public int MaxJsonLength { get; set; }
    public int RecursionLimit { get; set; }
    public int MaxSerializationDepth { get; set; }

    public override void ExecuteResult(ControllerContext context)
    {
      if (context == null)
      {
        throw new ArgumentNullException("context");
      }

      if (JsonRequestBehavior == JsonRequestBehavior.DenyGet &&
          String.Equals(context.HttpContext.Request.HttpMethod, "GET", StringComparison.OrdinalIgnoreCase))
      {
        throw new InvalidOperationException(JsonRequest_GetNotAllowed);
      }

      var response = context.HttpContext.Response;

      if (!String.IsNullOrEmpty(ContentType))
      {
        response.ContentType = ContentType;
      }
      else
      {
        response.ContentType = "application/json";
      }

      if (ContentEncoding != null)
      {
        response.ContentEncoding = ContentEncoding;
      }

      if (Data != null)
      {
        var serializer = new JavaScriptSerializer
        {
          MaxJsonLength = MaxJsonLength,
          RecursionLimit = RecursionLimit
        };

        serializer.RegisterConverters(new List<JavaScriptConverter> { new EFJsonConverter(MaxSerializationDepth) });

        response.Write(serializer.Serialize(Data));
      }
    }

Balsamiferous

Looks good :) I'll take a closer look at the code soon. I think there are some performance/logical improvements to be made (from having written similar code before I went for my alternative approach). If you'd like, I can give you some feedback later, or edit the code and let you decide if you want to roll it back. Also, what is RelationshipManager? Should that be removed? – Faraday

Yes, please do provide performance feedback/suggestions. In my case its part of one off/small scale object serialization so perf is not critical. 'RelationshipManager' I believe is a property that gets added to the EF4 DynamicProxies (maybe only in POCO mode), if you remove that line it will end up serializing that property into your JSON which you probably do not want. – Balsamiferous

I was thinking that you should be able to pass in the supported types, and also be able to pass in a list of deliberately ignored types (types it "handles", then ignores). The list of supported types could be generated by a T4 template if you're generating your types off the model, or is somewhat easy to create manually. Also, getting the hash code alone isn't enough to establish uniqueness. You have to check Equals too. I suggest using HashSet<object> with an IEqualityComparer<object> that returns the normal hash code, and checks reference equality rather than calling Equals. – Faraday

Also, I think the maxDepth is ignored on supported types, since you create a new serializer when you recurse. Not sure how much use there is to the depth-limiting code at that point. Maybe you should remove that, and instead of creating a new instance, just pass the one you got in Serialize. – Faraday

Actually no Merlyn. The new serializer that is created is purely to satisfy the interface, if you look closely, at no point does it use the passed in serializer parameter, it is only using maxDepth from the EFJavaScriptConverter js. – Balsamiferous

If that is the case, then you can pass in the existing one safely, tho you're right that it is a nit-pick at that point. I will make some changes on the code and put it up on a paste-bin for you later and let you decide if you like the changes. – Faraday

Answer 2

You can also detach the object from the context and it will remove the navigation properties so that it can be serialized. For my data repository classes that are used with Json i use something like this.

 public DataModel.Page GetPage(Guid idPage, bool detach = false)
    {
        var results = from p in DataContext.Pages
                      where p.idPage == idPage
                      select p;

        if (results.Count() == 0)
            return null;
        else
        {
            var result = results.First();
            if (detach)
                DataContext.Detach(result);
            return result;
        }
    }

By default the returned object will have all of the complex/navigation properties, but by setting detach = true it will remove those properties and return the base object only. For a list of objects the implementation looks like this

 public List<DataModel.Page> GetPageList(Guid idSite, bool detach = false)
    {
        var results = from p in DataContext.Pages
                      where p.idSite == idSite
                      select p;

        if (results.Count() > 0)
        {
            if (detach)
            {
                List<DataModel.Page> retValue = new List<DataModel.Page>();
                foreach (var result in results)
                {
                    DataContext.Detach(result);
                    retValue.Add(result);
                }
                return retValue;
            }
            else
                return results.ToList();

        }
        else
            return new List<DataModel.Page>();
    }

Pedant

I actually like this approach but sometimes I need to be able to serialize more than 1 level deep – Balsamiferous

This was exactly what I was looking for - at least the DataContext.Detach() method. This got rid of the 'Object disposed' exception... – Overstretch

In EF6 after detaching by setting the EntityState it will have a 2 level depth. – Rubbico

Answer 3

I have just successfully tested this code.

It may be that in your case your Message object is in a different assembly? The overriden Property SupportedTypes is returning everything ONLY in its own Assembly so when serialize is called the JavaScriptSerializer defaults to the standard JavaScriptConverter.

You should be able to verify this debugging.

Balsamiferous

+1; I had this problem with the code... I provide a work around in my answer. Though no code. I threw it out since I went for a solution that involved projecting onto new types. – Faraday

Answer 4

Your error occured due to some "Reference" classes generated by EF for some entities with 1:1 relations and that the JavaScriptSerializer failed to serialize. I've used a workaround by adding a new condition :

    !p.Name.EndsWith("Reference")

The code to get the complex properties looks like this :

    var complexProperties = from p in type.GetProperties()
                                    where p.CanWrite &&
                                          p.CanRead &&
                                          !p.Name.EndsWith("Reference") &&
                                          !_builtInTypes.Contains(p.PropertyType) &&
                                          !_processedObjects.Contains(p.GetValue(obj, null)
                                            == null
                                            ? 0
                                            : p.GetValue(obj, null).GetHashCode())
                                    select p;

Hope this help you.

Carpology

Answer 5

I had a similar problem with pushing my view via Ajax to UI components.

I also found and tried to use that code sample you provided. Some problems I had with that code:

• SupportedTypes wasn't grabbing the types I needed, so the converter wasn't being called
• If the maximum depth is hit, the serialization would be truncated
• It threw out any other converters I had on the existing serializer by creating its own new JavaScriptSerializer

Here are the fixes I implemented for those issues:

Reusing the same serializer

I simply reused the existing serializer that is passed into Serialize to solve this problem. This broke the depth hack though.

Truncating on already-visited, rather than on depth

Instead of truncating on depth, I created a HashSet<object> of already seen instances (with a custom IEqualityComparer that checked reference equality). I simply didn't recurse if I found an instance I'd already seen. This is the same detection mechanism built into the JavaScriptSerializer itself, so worked quite well.

The only problem with this solution is that the serialization output isn't very deterministic. The order of truncation is strongly dependent on the order that reflections finds the properties. You could solve this (with a perf hit) by sorting before recursing.

SupportedTypes needed the right types

My JavaScriptConverter couldn't live in the same assembly as my model. If you plan to reuse this converter code, you'll probably run into the same problem.

To solve this I had to pre-traverse the object tree, keeping a HashSet<Type> of already seen types (to avoid my own infinite recursion), and pass that to the JavaScriptConverter before registering it.

Looking back on my solution, I would now use code generation templates to create a list of the entity types. This would be much more foolproof (it uses simple iteration), and have much better perf since it would produce a list at compile time. I'd still pass this to the converter so it could be reused between models.

My final solution

I threw out that code and tried again :)

I simply wrote code to project onto new types ("ViewModel" types - in your case, it would be service contract types) before doing my serialization. The intention of my code was made more explicit, it allowed me to serialize just the data I wanted, and it didn't have the potential of slipping in queries on accident (e.g. serializing my whole DB).

My types were fairly simple, and I didn't need most of them for my view. I might look into AutoMapper to do some of this projection in the future.

Faraday

I agree that projection is a safer approach even though it may require more code but to not have to worry about serialization is quite nice. You might try experimenting with the code I posted because I had exactly the same issues as you and I believe my solution works without having to needlessly project everything. – Balsamiferous

@Tom: Yes, I think having both options available is a good thing. Often my view doesn't match my view model, and I want to remove unimportant data from being accessible from the client, so projection makes sense. Other times, I don't have those requirements, and my model is small and most of its data is relevant, so serialization makes sense. – Faraday