1

We have a local Web API used for internal clients to retrieve data. We use application/json to transport the data across the Web API.

The retrieved JSON data has a simple structured class 'Response'. This contains a status enum, errorMessage string, errorDetail string and an object. This object always holds a list of requested class objects, being models for our MVVM UIs.

When retrieving many thousand class objects, we found deserialization, using Newtonsoft.Json slow as we had to deserialize into the 'Response' class and also deserialize the Response.Object.

The way we sped up the process was to create a Reflection.Emit.ModuleBuilder class as shown here:

public class ResponseBuilder
{
    private static ModuleBuilder _ModuleBuilder;
    private static Dictionary<string, Type> _ResponseTypes = new Dictionary<string, Type>();
    private static object _lock = new object();

    /// <summary>
    /// Create a new type 'Response_T' based on Data.Response with Object set to List<T>
    /// </summary>
    public static Type CreateResponseType<T>()
    {
        lock (_lock)
        {
            if (_ModuleBuilder == null)
            {
                AssemblyName aname = new AssemblyName("ResponseBuilder");
                AppDomain currentDomain = AppDomain.CurrentDomain;
                AssemblyBuilder abuilder = currentDomain.DefineDynamicAssembly(aname,
                                           AssemblyBuilderAccess.Run);
                _ModuleBuilder = abuilder.DefineDynamicModule("ResponseClasses");
            }

            // Set new type name to 'Response_T'
            string newTypeName = "Response_" + typeof(T).FullName;

            // Check if this type has already been created and stored in dictionary
            if (_ResponseTypes.ContainsKey(newTypeName))
            {
                return _ResponseTypes[newTypeName];
            }

            // Create the new type
            TypeBuilder typeBuilder = _ModuleBuilder.DefineType(newTypeName,
                TypeAttributes.Public
                | TypeAttributes.Class
                | TypeAttributes.AutoClass
                | TypeAttributes.AnsiClass
                | TypeAttributes.ExplicitLayout);

            int fieldOffset = 0;

            FieldBuilder fieldBuilder = typeBuilder.DefineField("Status", typeof(Data.Response.ResponseStatus), FieldAttributes.Public);
            fieldBuilder.SetOffset(fieldOffset);
            fieldOffset += IntPtr.Size;

            fieldBuilder = typeBuilder.DefineField("ErrorMessage", typeof(string), FieldAttributes.Public);
            fieldBuilder.SetOffset(fieldOffset);
            fieldOffset += IntPtr.Size;

            fieldBuilder = typeBuilder.DefineField("ErrorDetails", typeof(string), FieldAttributes.Public);
            fieldBuilder.SetOffset(fieldOffset);
            fieldOffset += IntPtr.Size;

            fieldBuilder = typeBuilder.DefineField("Logon", typeof(Data.Logon), FieldAttributes.Public);
            fieldBuilder.SetOffset(fieldOffset);
            fieldOffset += IntPtr.Size;

            fieldBuilder = typeBuilder.DefineField("Object", typeof(List<T>), FieldAttributes.Public);
            fieldBuilder.SetOffset(fieldOffset);
            fieldOffset += IntPtr.Size;

            // Add newly create type to static list
            Type type = typeBuilder.CreateType();
            _ResponseTypes.Add(newTypeName, type);

            return type;
        }
    }
}

After getting the HttpResponseMessage from the Web API call, we deserialize the json response as follows:

Response response = new Response();
Type dynamicResponseType = ResponseBuilder.CreateResponseType<T>();
JsonSerializer serializer = new JsonSerializer();
System.IO.Stream stream = httpresponse.Content.ReadAsStreamAsync().Result;
using (System.IO.StreamReader sr = new System.IO.StreamReader(stream))
{
    using (JsonTextReader jtr = new JsonTextReader(sr))
    {
        var x = serializer.Deserialize(jtr, dynamicResponseType);
        FieldInfo fi = dynamicResponseType.GetField("Status");
        response.Status = (Response.ResponseStatus)fi.GetValue(x);
        fi = dynamicResponseType.GetField("ErrorMessage");
        response.ErrorMessage = (string)fi.GetValue(x);
        fi = dynamicResponseType.GetField("ErrorDetails");
        response.ErrorDetails = (string)fi.GetValue(x);
        fi = dynamicResponseType.GetField("Logon");
        response.Logon = (Logon)fi.GetValue(x);
        fi = dynamicResponseType.GetField("Object");
        response.Object = fi.GetValue(x);
    }
}

This works perfectly and is quick at creating the List<T> response.Object.

Now, on to my issue. I am migrating away from Newtonsoft.Json to System.Text.Json in .NET Framework 4.8. I am using System.Text.Json version 9.0.6 nuGet package.

This is the last part of Newtonsoft code I need to migrate and I am having issue with deserializing directly into my dynamicResponseType.

JsonTextReader is not available in System.Text.Json, I have tried playing around with Uft8JsonReader but this doesn't seem to help.

I have tried:

var x = JsonSerializer.Deserialize(httpresponse.Content.ReadAsStreamAsync().Result, dynamicResponseType);

x is instantiated, but all the properties are null.

Improve this question
2
  • 1
    TBH not sure why do you need dynamically generated type here. Define a type and deserialize to it and check the performance, and if performance is not suitable - only then look into the other approaches. System.Text.Json was designed with performance as one of main goals and it is fast enough in majority of cases that I know about. Also I'm very surprised that your current approach is faster than using the Newtonsoft.Json "idiomatically", but it's hard to tell (but it would be great to see the benchmarks and minimal reproducible example). Commented Jun 21 at 6:34
  • Interesting, I haven't got to the benchmark stage yet, still trying to replace all of the various methods in our project. I'll look forward to having some performance increases in changing to system.Text.Json Commented Jun 23 at 9:13

1 Answer 1

Reset to default
2

Here is a very simplified example that reproduces approximately your case. It works for me on Framework 4.8 with System.Text.Json v9.0.6 installed package

internal class Program
{
    static void Main()
    {
        var json = "{\"Message\": \"message text\"}";

        var stream = new MemoryStream(Encoding.UTF8.GetBytes(json));
        
        Type dynamicResponseType = CreateTestType();

        var options = new System.Text.Json.JsonSerializerOptions
        {
            IncludeFields = true
        };

        var x = System.Text.Json.JsonSerializer.Deserialize(stream, dynamicResponseType, options);

        FieldInfo fi = dynamicResponseType.GetField("Message");
        var message = (string)fi.GetValue(x);

        Console.WriteLine(message);
        Console.Read();
    }

    static Type CreateTestType()
    {
        var newTypeName = "TestType";

        AssemblyName aname = new AssemblyName("ResponseBuilder");
        AppDomain currentDomain = AppDomain.CurrentDomain;
        AssemblyBuilder abuilder = currentDomain.DefineDynamicAssembly(aname,
            AssemblyBuilderAccess.Run);
        var moduleBuilder = abuilder.DefineDynamicModule("ResponseClasses"); 

        // Create the new type
        TypeBuilder typeBuilder = moduleBuilder.DefineType(newTypeName,
            TypeAttributes.Public
            | TypeAttributes.Class
            | TypeAttributes.AutoClass
            | TypeAttributes.AnsiClass
            | TypeAttributes.ExplicitLayout);

        int fieldOffset = 0;

        var fieldBuilder = typeBuilder.DefineField("Message", typeof(string), FieldAttributes.Public);
        fieldBuilder.SetOffset(fieldOffset);

        return typeBuilder.CreateType();
    }
}

It seems your main problem is that System.Text.Json serializer with default settings does not deserialize fields but only properties. To enable field serialization, do this

var options = new System.Text.Json.JsonSerializerOptions
{
    IncludeFields = true
};

So, going back to your code, it seems to me that this should work

Response response = new Response();
Type dynamicResponseType = ResponseBuilder.CreateResponseType<T>();
var options = new System.Text.Json.JsonSerializerOptions
{
    IncludeFields = true
};
System.IO.Stream stream = httpresponse.Content.ReadAsStreamAsync().Result;
var x = System.Text.Json.JsonSerializer.Deserialize(stream, dynamicResponseType, options);

FieldInfo fi = dynamicResponseType.GetField("Status");
response.Status = (Response.ResponseStatus)fi.GetValue(x);
fi = dynamicResponseType.GetField("ErrorMessage");
response.ErrorMessage = (string)fi.GetValue(x);
fi = dynamicResponseType.GetField("ErrorDetails");
response.ErrorDetails = (string)fi.GetValue(x);
fi = dynamicResponseType.GetField("Logon");
response.Logon = (Logon)fi.GetValue(x);
fi = dynamicResponseType.GetField("Object");
response.Object = fi.GetValue(x);
Improve this answer
Sign up to request clarification or add additional context in comments.

1 Comment

That is brilliant, thank you. I was so close. I knew fields were not included by default and, looking back, I can see my ResponseBuilder class defines fields but I hadn't connected the two! Obviously the end of a very long week.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.