Sergey Maskalik

Sergey Maskalik's blog

In the pursuit of mastery

JSON.NET is a great library for serializing objects to and from json strings. In case you need to have a more control of how your object is being serialized this post covers creation of custom json converter. For instance, I came across a scenario where a json result had to have a property name starting with a $(dollar sign) like “$and” and as you may guess properties in .NET cannot start with a dollar sign. So that’s where we would need a custom json converter.

Setup

First thing you need to do is to create a custom class the derives from JsonConverter, and override 3 methods. In the CanConvert method we check if the passed in type can be assigned to our target type WeirdName.

public class WeirdNameSerializer : JsonConverter
{
    public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
    {
        throw new NotImplementedException();
    }

    public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
    {
        throw new NotImplementedException();
    }

    public override bool CanConvert(Type objectType)
    {
        return typeof(WeirdName).IsAssignableFrom(objectType);
    }
}

Next is you need to decorate your class which will be serialized with the attribute of a newly created type

[JsonConverter(typeof(WeirdNameSerializer))]
public class WeirdName
{
    public string Name { get; set; }
    public string Value { get; set; }
}

Custom Writing

Now in your WriteJson method we will cast the value object to your serilized class so we can access properties and do our custom serialization on them.

public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
{
    var name = value as WeirdName;
    writer.WriteStartObject();
    writer.WritePropertyName("$" + name.Name);
    serializer.Serialize(writer, name.Value);
    writer.WriteEndObject();
}

You can also nest objects within other objects

    var name = value as WeirdName;
    writer.WriteStartObject();
    writer.WritePropertyName("$" + name.Name);
    writer.WriteStartObject();
    writer.WritePropertyName("nested");
    serializer.Serialize(writer, name.Value);
    writer.WriteEndObject();
    writer.WriteEndObject();

Custom Reading

First thing is you need to load the json reader into a JObject. Then you can access fields if you know the key of the property with jsonObject[“fieldName”]. If the schema is the same you can use

var name = new WeirdName(); 
serializer.Populate(jObject.CreateReader(), name); 

to populate properties automatically. However in our case our property name is not standard so we’ll have to manually get it from list of properties.

public override object ReadJson(JsonReader reader, Type objectType, 
object existingValue, JsonSerializer serializer)
{
    JObject jsonObject = JObject.Load(reader);
    var properties = jsonObject.Properties().ToList();    
    return new WeirdName { 
                           Name = properties[0].Name.Replace("$",""), 
                           Value = (string)properties[0].Value 
                         };
}

Testing our custom serializer

// Arrange
var weird = new WeirdName {Name = "first", Value = "Sergey"};

// Act
// weird gets serialized into "{\"$first\":\"Sergey\"}"
var result = JsonConvert.SerializeObject(weird); 

var test = JsonConvert.DeserializeObject<WeirdName>(result);

// Assert
Assert.AreEqual(weird.Name,test.Name);
Assert.AreEqual(weird.Value,test.Value);

And that’s it, now you should have an idea of how to implement custom json serializers. Please post if you have any questions.

Update 08/07/2012

For latest documentation please see factual-csharp-project on github.

Introduction

FactualDriver is a .NET client library for consuming Factual API. It abstracts creation of queries, oAuth header signing and synchronous web request/response process. It’s easy to use and you can get a response from factual in 2 lines of code. It also exposes a generic access to signed HttpWebRequest if you need more granular control or if you decide that you need to call Factual in asynchronous matter.

Getting started

From you nuget package manager console install FactualDriver package

PM> Install-Package FactualDriver

Next, you create an instance of the factual driver with your consumer key and consumer secret key that you received from Factual. And the simplest way to get data is to use a RawQuery function which will take table name as the first parameter and your raw url query and return you a json result string.

Factual client = new Factual("YourConstumerKey","YourConsumerSecret");
string jsonResponse = client.RawQuery(""t/global", "filters={\"name\":\"Star\"}");

As an option you can use included JSON.NET parser to convert response into a C# dynamic object.

dynamic json = JsonConvert.DeserializeObject(jsonResponse);
string status = (string)json.status;

Filters

Building query string by hand is error prone, hard to maintain and just not pretty. As you might have noticed, Factual API uses json serialized object in the query string for filters, and uses a simple key value pair for other parameters like limit, or include_count. You can see a full list of parameters here.

Rather than hacking string together, driver provides an object oriented way of creating parameters. IFilter classes provide a structure for parameters and are flexible enough to work with future api changes. Let’s take a look at first filter..

RowFilter

Constructor Signature:

RowFilter(string fieldName, string comparisonOperator, object compareToValue)

You create a row filter by providing a field name, operator and an object value. The reason compareToValue is an object is because you should be able to provide a simple type like integer as well as an array that will get serialized into json. For a list of field and operators see factual documentation.

Here is how we create a row filter that instructs api to return all record where locality is not Los Angeles and Northridge and execute a query.

var filter = new RowFilter("locality", "$nin", new [] { "Los Angeles", "Northridge"});
string result = Factual.Query("t/global", filter);

The query will get serialized into {"locality":{"$nin":["Los Angeles","Santa Monica"]}}, and you have compiler checking,intellisense, and no hacking together curly braces.

If you filter is a simple equal you can use a shorthand constructor:

public RowFilter(string fieldName, object compareToValue). 

Which will result in something like this {“region”:”CA”}, which is acceptable shorthand according to factual documentation.

Let’s see another example where use string rather than array to filter data.

var filter = new RowFilter("name", "$search", "Charles");  Outputs:`{"name":{"$search":"Charles"}}`

GeoFilter

Geofilter is very similar in terms of usage to RowFilter, except you provide different parameters of latitude and longitude along with radius distance in meters.

Signature:

GeoFilter(decimal latitude, decimal longitude, int distance)

Usage:

var geoFilter = new GeoFilter(34.06018m, -118.41835m, 5000);
string result = Factual.Query("t/restaurants-us", geoFilter);

You also have an option to override shape of the filter, the default shape is $circle by setting GetFilter’s Shape property. As well as override distance units by setting DistanceUnits property. However according to the documentation there are no other shapes available as of yet, so this might be useful in the future.

SearchFilter

Search filter only takes one parameter which is the search string. Here is an example:

var filter = new SearchFilter("Vegan,Los Angeles");
string result = Factual.Query("t/restaurants-us", filter);

Filter - Other parameters - Combining FIlters

There are other parameters like limit and offset and sort etc. Those are sent to factual as basic key value pair and not a serialized json object. There is also a wrapper for those properties, Filter. Here is an example of setting a limit on the query and combining it with RowFilter.

var limitFilter= new Filter("limit", "20");
var rowFilter = new RowFilter("name", "$bw", "Star");
string result = Factual.Query("t/restaurants-us", rowFilter, limitFilter);

ConditionalRowFilter

There is also a conditional operators that allows you to set AND or OR operator and group your row filters together.

var filter = new ConditionalRowFilter("$and", new[]
             {
               new RowFilter("name", "$search", "McDonald's"),
               new RowFilter("category", "$bw", "Food & Beverage")
             });

First parameter takes a string operator value $and or $or and second takes a collection of RowFilter types. Result will get serialized into an api query like this:

"filters={\"$and\":[{\"name\":{\"$search\":\"McDonald's\"}},{\"category\":{\"$bw\":\"Food & Beverage\"}}]}"

#HttpWebRequest Factual.Query or Factual.RawQuery will create a synchronous web request and will also synchronously get a response back from factual, read it as a string and return to the caller. If you have a high traffic environment that’s probably not what you going to want because the number of process threads is limited and when you call outside service synchronously it uses it up a thread, and if you have more users than your threads then your application will come to a screeching halt. So if you are planning on experiencing a very high traffic, you want to call factual api asynchronously. And you can do that by getting an HttpWebRequest object directly from Factual.CreateWebRequest(string query) and then executing the request asynchronously.

CreateWebRequest accepts a raw url query but you are also not on your own here. You can create your filters and then call JsonUtil.ToQueryString and it will serialize a collection of filters into correct json query string.

That’s it for now, please let me know if you have any questions or if something I missed.

Browsers cache pages by default in order to quickly serve the content when back button is clicked. There are times however when you don’t want that functionality. For example if your user logs out you don’t want them to be able to press back button and navigate back to member page again. Or other scenarios where javascript updates shopping cart on the page, but when you hit back button on the page browser serves up content from cache which doesn’t have the new item count in the cart.

Originally I’ve stumbled upon method below, but soon to discovered that it only works for some browsers including Internet Explorer.

   Page.Response.Cache.SetCacheability(HttpCacheability.ServerAndNoCache);

Which results in the following header attribute, which is enough for IE but not enough for other browsers like Firefox and Chrome.

   Cache-Control:no-cache

After doing more research on the browser cache headers I found this comprehensive guide that explains that no-cache does not necessarily tells the browser not to cache results:

The “no-cache” directive, according to the RFC, tells the browser that it should revalidate with the server before serving the page from the cache. … In practice, IE and Firefox have started treating the no-cache directive as if it instructs the browser not to even cache the page. We started observing this behavior about a year ago. We suspect that this change was prompted by the widespread (and incorrect) use of this directive to prevent caching.directive to prevent caching.

We have used that header incorrectly just like the article said. And that’s why we are seeing the random results for different browsers because originally the “no-cache” header is not the correct one to instruct browser not to cache the content. I’m guilty of this myself since when I first encountered this problem I quickly found the solution online and was happy to move on with my life, without trying to fully understand the intent of the Cache Control headers.

After reading through and understanding what each header suppose to do it was clear that the correct header to instruct browsers not to cache pages is the Cache Control: No-store. In addition it’s a good practice to include Cache Control: No-cache just to be on the safe side. And for sensitive content I think it’s also good to tell proxies not to cache the http result for example account areas. And to do that we need to set Cache-Control: private.

In ASP.NET we have the following method

Response.Cache.SetNoStore();

Which sets headers to Cache-Control:private, no-store. That takes care almost everything and the only thing left to do is to append the recommended no-cache to be on the safe side

Response.Cache.AppendCacheExtension("no-cache");

And now we have the desired headers: Cache-Control:private, no-store, no-cache

Please let me know if you find any situations where above code doesn’t work. I’d like nail this problem once and for all.

Cheers

Update 04-12-2012

I’ve read another good resource on caching and think that adding expires header would also be beneficial for some other edge cases. So it wouldn’t hurt to add to this to your code as well:

Response.Expires = 0;

That brings us up to the three call that we need to make to make sure the page is not being cached:

Response.Cache.SetNoStore();
Response.Cache.AppendCacheExtension("no-cache");
Response.Expires = 0;