This time I’ll talk about performance of serialization. Serialization is a common task we use mostly for communication and storage. This post will give a wide serialization performance comparison.
Real life scenarios- Lately a micro services architecture become very common, in such architecture you have to provide a way for your micro services to communicate between themselves so you’ll have to serialize your objects for that. Normally you will use messaging frameworks that will do it for you but it’s essential to understand what’s going under the hood. In other cases, you’ll need to develop a framework or a tool that will need to serialize it’s objects itself. In addition, in many communication frameworks you’re able to change the serialization method so it’s important to understand what you’ll achieve by doing so.
In this post I won’t talk about the advantages or disadvantages of each format/framework, I’ll stick strictly with performance (Space and Speed). That said, when you choose a format and framework you sure need to think about much more than that. Examples are ease of use, extensibility, flexibility, versioning, and much more.
Although my test run in C#, this post applies to any technology. The size obviously will be the same, and about the speed I believe that it will differ, but the ratio between the formats will be roughly the same in most cases.
I’ll test popular formats and frameworks –
XML (XmlSerializer, DataContractSerializer) , BinaryFormatter – Are included with the .NET framework http://msdn.microsoft.com/en-gb/vstudio/hh341490.aspx
JSON – http://james.newtonking.com/json, https://servicestack.net/text
MsgPack – https://github.com/msgpack/msgpack-cli
Protobuf – https://code.google.com/p/protobuf-net/
The way I built the testing code it’s very easy to set up new tests for other frameworks. You just need to implement the abstract methods of the SerializationTester<TTestObject> class. So with the provided code you can easily test different serialization frameworks with different sample data that fits your needs the best. This is something you can do within minutes with my code.
So why do I need to change the serialization method in my application? In most applications you probably don’t.
However, it’s important to be aware of the differences which are significant. If your application is heavy on serialization you should consider it. It’s important to understand that we’re testing two factors –
Space which affects storage/memory (storing objects) and network usage(communication)
Speed which the time takes to serialize/deserialize the object.
So let’s begin with the comparison. To compare we need data, I’ve chosen to use a list of Belgian beers, after all – who doesn’t like beer? So by Wikipedia there’re 1610 different beers made in Belgium and they all listed on this page – http://en.wikipedia.org/wiki/List_of_Belgian_beer.
Then I run the tests, and plotted them on a chart for you. Of course the code to retrieve the list and all the tests is available on GitHub – https://github.com/maximn/SerializationPerformanceTest_CSharp
How a Beer looks like?
It’s pretty straight forward, a beer has – Brand, Brewery, Alcohol level and a List of sorts that apply to this beer.
I wanted to do the comparison for large data and small data so I used the list of all 1610 beers and one beer respectively.
This is the summary of the results (Size in bytes, time in milliseconds).
But of course I’ve added charts to make the comparison easy
(* smaller is better)
We can see huge differences between formats. Interesting to note that Binary which would be in the middle for large objects would be the worst for small objects.
Now for the speed, for easy comparison I’ve decided to normalize the results so in both charts it shows the time for 1 item (beer). For the large data (list) I just used the time took to handle the list divided by the number of items(1610). I’ve ran the test 100 times and took the average run speed.
(* smaller is better)
All the test ran on my laptop – Lenovo X1 Carbon (i7-3667U). The tests obviously gave me different results each run but what’s really interesting is the ratio between the formats.
- It is important to say that I’ve used all those frameworks with their default settings. I’m sure it’s possible to tweak it a bit but I just wanted to compare their defaults.
A bit about the testing code – All testers implement SerializationTester<TTestObject> class. They implement an Init, Serialize, and Deserialize methods. The base tester will run the tests 100 times and will output the results to the console. All the tests measure in memory serialization/deserialization so hard disk speed doesn’t affect the results.
So which one should you use? I don’t really have an answer for that one. It all depends on your needs, but this post intends to help you make the right decision.