Both – XML and JSON formatters employed?

Topics: Web Api
Jan 30, 2012 at 9:48 PM

I have a very strange problem that I can’t figure out. I am using jquery to POST to my WebAPI-based service.

I am using JSON and both Accept and Content-Type headers are set to “application/json”. On a server side my load is de-serialized into type that contains Dictionary-based property. JSON has no problem de-serializing it (both, built-in and JSON.NET work fine). However, I am getting error “Cannot serialize member ... System.Collections.Generic.Dictionary`2 ... because it implements IDictionary.” Call stack shows that it comes from XmlSerializer, despite me requesting only JSON serialization (and why is it doing serialization on a receiving part – shouldn't it de-serialize?)

If offending Dictionary property is removed – then everything seems to work just fine.

If I remove XmlMediaTypeFormatter from configuration.Formatters collection – then error goes away and I am getting my Dictionary deserialized. However, I suspect that doing so will prevent me from ever using XML formatting if I need to.

The question is – why XML formatter is even coming to a play?

Here is a fragment from what Fiddler shows:

POST http://localhost:9500/DataService/GetData HTTP/1.1
Accept: application/json
Content-Type: application/json; charset=utf-8
X-Requested-With: XMLHttpRequest
Referer: http://localhost:9500/Form1.aspx
Accept-Language: en-us
Accept-Encoding: gzip, deflate
User-Agent: Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)
Host: localhost:9500
Content-Length: 366
Connection: Keep-Alive
Pragma: no-cache

Thanks,

Sam

Jan 30, 2012 at 11:03 PM

Can you put all of your operations into the thread, sounds like you might be messing something up.

Jan 31, 2012 at 3:41 PM

I am not sure what operations to put in - at this time I have very simple project that I am using to test Web API with different datatypes.

Here is what I am doing in a global.asax.cs

 public static void RegisterRoutes(RouteCollection routes)
 {
            routes.IgnoreRoute("{resource}.axd/{*pathInfo}");

            var configuration = new WebApiConfiguration(){EnableTestClient = true};

            // Uncomment to use JSON.NET
            //configuration.Formatters.Insert(0, new NewtonsoftJsonFormatter());

            routes.SetDefaultHttpConfiguration(configuration);
            routes.MapServiceRoute<BooksService>("DataService");

            routes.MapRoute(
                "Default", // Route name
                "{controller}/{action}/{id}", // URL with parameters
                new { controller = "Home", action = "Index", id = UrlParameter.Optional } // Parameter defaults
            );

}

This is my service code

using System;
using System.Collections.Generic;
using System.Linq;
using System.Runtime.Serialization;
using System.ServiceModel;
using System.ServiceModel.Activation;
using System.ServiceModel.Web;
using System.Text;
using System.Web.Routing;
using Microsoft.ApplicationServer.Http;
using System.Json;
using System.Net.Http;
using System.Net.Http.Formatting;
using System.IO;
using System.Net.Http.Headers;
using Newtonsoft.Json;


namespace BaseApp.Server
{
    [ServiceContract]
    [AspNetCompatibilityRequirements(RequirementsMode = AspNetCompatibilityRequirementsMode.Required)]
    public class BooksService
    {
        [JsonObject]
        public class MyClass
        {
            [JsonProperty]
            [JsonConverter(typeof(Newtonsoft.Json.Converters.IsoDateTimeConverter))]
            public DateTime? DateArg { get; set; }

            [JsonProperty]
            public string StringArg { get; set; }
        }

        public class MyRequest
        {
            [JsonProperty]
            public string Data { get; set; }

            [JsonProperty]
            public MyClass ClassProp { get; set; }
           
            [JsonProperty]
            public MyClass[] ClassArray { get; set; }

            [JsonProperty]
            [JsonConverter(typeof(Newtonsoft.Json.Converters.IsoDateTimeConverter))]
            public DateTime MyDate { get; set; }

            [JsonProperty]
            public Dictionary<string,object> MyDictionary { get; set; }

            [JsonProperty]
            public List<object> MyList { get; set; }

            [JsonProperty]
            public List<MyClass> ClassList { get; set; }

            [JsonProperty]
            public long MyInt { get; set; }

            [JsonProperty]
            public string Type { get; set; }
        }


        [WebInvoke(Method = "POST", UriTemplate = "GetByJsonC")]
        public HttpResponseMessage<MyRequest> GetByJsonC(MyRequest data)
        {
            HttpResponseMessage<MyRequest> resp = new HttpResponseMessage<MyRequest>(BuildSampleData());
            return resp;
        }
        

        protected MyRequest BuildSampleData()
        {
                MyRequest parms = new MyRequest();
                parms.Type = "ActionType";
                parms.Data = DateTime.Now.Date.ToString();
               parms.MyDate = DateTime.Now;
                parms.MyInt = 123;

                parms.ClassProp = new MyClass ();
                parms.ClassProp.DateArg = DateTime.Now;
                parms.ClassProp.StringArg = "Dictionary entry - XYZ";

                parms.ClassArray = new MyClass[3];
                parms.ClassArray[0] = new MyClass();
                parms.ClassArray[0].StringArg = "Class Array XYZ";
                parms.ClassArray[0].DateArg = DateTime.Now;

                parms.ClassArray[1] = new MyClass();;
                parms.ClassArray[1].StringArg = "Class Array ABC";

                parms.ClassList = new List<MyClass> ();
                parms.ClassList.Add(new MyClass());
                parms.ClassList[0].StringArg = "List XYZ";
                parms.ClassList.Add(new MyClass());

                parms.ClassList[1].StringArg = "List ABC";

                parms.MyList = new List<object>();
                parms.MyList.Add(15);
                parms.MyList.Add( DateTime.Now);

                parms.MyDictionary = new Dictionary<string,object>();
                parms.MyDictionary["StringArg"] = "String argument";
                parms.MyDictionary["IntArg"] = 111;
                parms.MyDictionary["DateArg"] =  DateTime.Now;

                return parms;
        }
    }

    public class NewtonsoftJsonFormatter : MediaTypeFormatter
    {
        public NewtonsoftJsonFormatter()
        {
            this.SupportedMediaTypes.Add(new MediaTypeHeaderValue("application/json"));
            this.SupportedMediaTypes.Add(new MediaTypeHeaderValue("text/json"));
        }

        protected override object OnReadFromStream(Type type, System.IO.Stream stream, HttpContentHeaders contentHeaders)
        {
            var serializer = new Newtonsoft.Json.JsonSerializer();
            using (var sr = new StreamReader(stream))
            using (var reader = new Newtonsoft.Json.JsonTextReader(sr))
            {
                var result = serializer.Deserialize(reader, type);
                return result;
            }
        }

        protected override void OnWriteToStream(Type type, object value, System.IO.Stream stream, HttpContentHeaders contentHeaders, System.Net.TransportContext context)
        {
            var serializer = new Newtonsoft.Json.JsonSerializer();

            using (var sw = new StreamWriter(stream))
            using (var writer = new JsonTextWriter(sw))
            {
                serializer.Serialize(writer, value);
            }
        }
    }

}

 

This is jQuery code

$(document).bind("ready", function ()
{
    jQuery.support.cors = true;
    myInit();
});

function myInit()
{
    $('#runIt').live('click', function ()
    {
        test();
    });
}

function test()
{
    var parms = {};
    parms.Type = "ActionType";
    parms.Data = new Date();
    parms.MyInt = 123;

    parms.ClassProp = {};
    parms.ClassProp.StringArg = "Dictionary entry - XYZ";

    parms.ClassArray = new Array();
    parms.ClassArray[0] = {};
    parms.ClassArray[0]["StringArg"] = "Class Array XYZ";
    parms.ClassArray[1] = {};
    parms.ClassArray[1]["StringArg"] = "Class Array ABC";

    parms.ClassList = new Array();
    parms.ClassList[0] = {};
    parms.ClassList[0]["StringArg"] = "List XYZ";
    parms.ClassList[1] = {};
    parms.ClassList[1]["StringArg"] = "List ABC";

    parms.MyList = new Array();
    parms.MyList[0] = "ABC";
    parms.MyList[1] = 15;
    parms.MyList[2] = new Date();

    parms.MyDictionary = {};
    parms.MyDictionary["StringArg"] = "String argument";
    parms.MyDictionary["IntArg"] = 111;
    parms.MyDictionary["DateArg"] = new Date();


    $.ajax({
        beforeSend: function (xhrObj)
            {
                xhrObj.setRequestHeader("Accept","application/json");
            },
        data: JSON.stringify(parms),
        cache: false,
        processData: false,
        type: "POST",
        dataType: "json",
        url: "DataService/GetByJsonC",
        contentType: "application/json; charset=utf-8",
        crossDomain: true,
        success: serverSuccess,
        error: serverError
    });

}

 

function serverSuccess(data)
{
    try
    {
        alert ("Success == " + data.Data);
    }
    catch (ex)
    {

    }
}

function serverError(data)
{
    alert("error === " + data.statusText);
}

 

 

Jan 31, 2012 at 4:34 PM

Do you understand that cross server ajax cannot pass things via POST?

Jan 31, 2012 at 7:27 PM

At this time I am not using CORS - the code is there for different testing.  Page and web services are on the same web site and in the same Visual Studio solution - I am getting the same issue with IIS and Development Server.

Also, for what it's worth - CORS works just fine with IE9 using POST - I was going to check further the issue with Chrome, but it is a topic for different discussion. I think you are confusing this with JSONP, which needs callbacks - I am using regular JSON.

Jan 31, 2012 at 7:37 PM

Completely off topic but "CORS" does not work in IE9 they have their own implementation... and that implementation is not in jquery. I was just wondering, because you are accepting a POST and you have cross-domain=true so you'd end up kicking yourself eventually anyway trying to do that.

Jan 31, 2012 at 7:39 PM
Edited Jan 31, 2012 at 7:40 PM

Back on topic. I think that newton thing is probably your problem. I am using JSON right now with absolutely no problems. Worked out of the box. Also the "service contract" stuff isn't necessary. Neither is the jsonobject stuff.

When doing an ajax call for a json response, use the method getJSON

http://api.jquery.com/jQuery.getJSON/

Jan 31, 2012 at 7:45 PM

Off topic again, jQuery.support is "read-only". Setting the values don't do anything.

Jan 31, 2012 at 7:50 PM
Edited Jan 31, 2012 at 8:05 PM

*deleted*

Jan 31, 2012 at 8:04 PM

It's the XML serializer. You have one of those in your formatters, its put in by default.

            var configuration = new WebApiConfiguration() { EnableTestClient = true };
            configuration.Formatters.Clear();
            configuration.Formatters.Add(new JsonpMediaTypeFormatter());
That'll remove all the formatters then only add jsonp

Jan 31, 2012 at 8:20 PM

Thank you for trying to help. I am well aware that Dictionary is an issue - this is what Xml serializer is complaining about. And it does not matter what types are implemented - it is enough that it is IDictionary.  JSON serializers do not care about it - both, built-in and JSON.NET work fine with either types. The only difference is in how they do it.

 My observations are as following:

- if I remove Xml formatter - then dictionaries are serialized with no problems, object or not.

- if I leave it (even moving down in a Formatters list) - it is invoked and is choking.

Why is it even called?

Regarding other notes:

- jQuery.support.cors - thank you - I just picked it from somewhere - was left there for further testing (does not hurt anything, anyway)

- CORS and IE9 - thank you again - totally outside of my area of expertise

- JsonObject stuff - this is for JSON.NET - you might want to check it out - highly recommended by everyone (even Glenn Block seems to prefer it over the built-in one - or so it seems :). Even if it is commented out - Xml Serializer is still erroring out.

Jan 31, 2012 at 8:25 PM

When the service starts up, it must build graphs or something of what will be serialized, you know, for caching I guess.

By default there are a few formatters added when using that configuration base class. One of them is XML.

Use the code I put in, or something like it, to remove the XML serializers and you will be fine. You just can't host XML.

var configuration = new WebApiConfiguration() { EnableTestClient = true };
            configuration.Formatters.Clear();
            configuration.Formatters.Add(new JsonpMediaTypeFormatter());

Jan 31, 2012 at 8:38 PM

I already know how to remove it, but I don't want to give up ability to host XML.

Anyone has any other ideas?

Jan 31, 2012 at 8:55 PM
Edited Jan 31, 2012 at 8:57 PM

Pretty sure the only other thing you can do is write your own XML serializer.

Or this maybe?

http://weblogs.asp.net/pwelter34/archive/2006/05/03/444961.aspx

Or implement

http://www.sharpserializer.com/en/index.html

Jan 31, 2012 at 9:15 PM

I can't believe this - one of the major points of Web Api was its ability to use the same service replying in different formats based on a content type.

If I remove Xml formatter in global.asax - then my entire web site will loose xml serialization capabilities. There got to be another solution.

Jan 31, 2012 at 9:17 PM

I think you should make a new thread that is titled more appropriately, or maybe make a post inside issue tracker, for the team to implement a different XML serializer instead of the built in one.

Feb 1, 2012 at 2:27 PM

I did a bit more digging and it looks like it is not serializer itself is invoked, but rather the system is checking if types are correct before invoking the service. And it is doing it for both - Xml and Json all the time.

I posted an issue inside the issue tracker as you suggested.

Feb 2, 2012 at 10:49 AM

Hi

 

You can try to use the DataContractSerializer instead, it's not as strict as the normal Xmlserializer:

 

config.Formatters.XmlFormatter.UseDataContractSerializer = true;

 

Feb 2, 2012 at 4:18 PM

That's a good idea. Although I've had issues with that before, can't recall what they were.

Feb 2, 2012 at 5:52 PM

This is an excellent idea - I can't believe I missed this.  We use DataContractSerializer all over with regular WCF, so we know how to deal with it.

I just checked - and it, indeed, is eliminating the error.

 

Thanks a lot.

Coordinator
Feb 2, 2012 at 7:02 PM
Edited Feb 2, 2012 at 7:29 PM

The issue I believe is that the xml formatter is pre-initialized inorder to do serialization. During that pre-init the body types exposed on the api are passed through the xml serializer whether you end up using the xml formatter or not.

There is a workaround though. You can override the serializer that we use for xml, even with a dummy one that does nothing. I have been meaning to try this for a while and I just did and it does work.

Below is a simple console app that demonstrates a class that serializes a dictionary. Notice how i override the formatter with a dummy serializer for a string. It will return an error if you choose application/xml but it works fine for json. I enabled the test client so you can test.

In either case this doesn't stop the xml formatter working for other scenarios where you want to validly support xml. I would go one step further though to remove the error at runtime (some weird status code is returned) and create a custom derived XmlFormatter overriding CanReadType and CanWriteType to be false for that set of types.

This way you don't get an error during init or during runtime.

using System;
using System.Collections.Generic;
using System.Linq;
using System.ServiceModel.Web;
using System.Text;
using System.Xml.Serialization;
using Microsoft.ApplicationServer.Http;

namespace ConsoleApplication1
{
    class Program
    {
        static void Main(string[] args)
        {
            var config = new WebApiConfiguration();
            config.Formatters.XmlFormatter.SetSerializer(typeof(HasDictionary), new XmlSerializer(typeof(string)));
            config.EnableTestClient = true;
            var host = new HttpServiceHost(typeof (HasDictionaryApi), config, "http://localhost:8080/foo");
            host.Open();
            Console.WriteLine("Listening");
            Console.ReadLine();
            host.Close();
        }
    }

    public class HasDictionaryApi
    {
        [WebGet]
        public HasDictionary Get()
        {
            return new HasDictionary();
        }
    }

    public class HasDictionary
    {
        public HasDictionary()
        {
            Dictionary = new Dictionary<string, string>();
            Dictionary["Foo"] = "Bar";
        }
        public Dictionary<string, string> Dictionary { get; set; } 
    }
}

 

Feb 2, 2012 at 10:50 PM

Interesting - I will have to digest that.

Actually, in my case - I do like the ability to use DataContractSerializer, because it fits nicely with our main Silverlight-based application that is using it exclusively. I might even re-use the same classes. I am not sure if we are going to use Xml with Web Api (JSON seems to be more than enough), but I did not want to cut that feature off either.

Thanks for the tip, though - I learned something new :)

BTW - as I mentioned earlier - I put the issue into the Issue Tracker.  I am not sure what the procedure is with that - I consider this particular issue solved for my case, since I have several work-arounds, but I don't know if you want to look into NOT checking types for the serializer that is not asked for.

Coordinator
Feb 2, 2012 at 10:59 PM

Yeah I think it is something we should explore or at least be able to easily turn it off without having to jump through hoops and create your own serializers :-)

Thanks

Glenn

Feb 21, 2012 at 1:43 PM

Hi

Just had the same problem.... I've been able to fix it by adding the [XmlIgnore] attribute to all my properties in the return object causing the problem....

 

Søren

Feb 21, 2012 at 2:09 PM

This, of course, makes it not serializable in XML. If you don't need XML serialization - then you can simply remove XML formatter altogether.

For me - using datacontract serialization did the trick. That was actually your own idea :)