Forward and backward compatibility in quasardb

I think the number one request from all our existing customers is to have as much as backward compatibility as possible with quasardb.

How it was done until now

Let’s be honest: in the 1.x series there was a lot of room for improvements, to say the least.

The quasardb protocol is in great part "meta-programmed", meaning that it’s written in C++ that itself generates C++ that will then be compiled (the exact technique is called Template Meta Programming, TMP for short).

The heart of the problem was that messages were part of a union that was analyzed at compile time to generate the appropriate serialization/deserialization code. It has the great advantage of generating very efficient code and being extremely safe.

The disadvantage however, is that adding a message could change the whole message serialization structure.

In other words, every time we added a new message or changed an existing message, the generated serialization would be incompatible with the previous one making backward and forward compatibility very difficult.

Fixing the problem

It’s very difficult to keep client and servers perfectly in sync on large scale deployments and this was a real problem.

Fortunately, we found a solution!

What we did is that we extracted the dispatching message part instead of relying on the union. Now messages register themselves at runtime with an unique identifier as you start your quasardb daemon. We made sure that this comes at no performance cost and actually managed to improve performance for some use case thanks to page locality.

That means that adding a message does not break anything and we now have both forward and backward compatibility.

The future

Starting with 2.0 quasardb offers backward and forward compatibility:

• When the client is more recent than the server, the server will answer “not supported” to new features but will be able to process older features, unless they were deprecated.
• When the client is older than the server, the server will be able to process all requests. The exact implementation of the requests may differ but the result will be understandable by the client.

Backward and forward compatibility can never be perfect because sometimes bug fixes or security enhancements require you to break retro-compatibility. We nevertheless believe we will be able to deliver a significantly easier upgrade path starting with 2.0, as you will no longer need to keep the client and servers perfectly in sync.