Optimized QuasarDB, part 1: storage

Posted by Leon Mergen on 19 Aug 2021

Reaching your performance, durability and efficiency goals in a production setup can be a challenge: there are about a million ways to deploy complex systems, and QuasarDB is no exception. In this series of blog posts, we will be taking you through the various aspects on how to implement a successful QuasarDB deployment in production.

Read More

Tags: optimization, storage

Don’t write your own persistence layer: why we chose RocksDB

Posted by Edouard on 04 Jun 2021

Our use case

In a nutshell, Quasar helps you anticipate the unpredicted.

Read More

Tags: c++, quasardb, rocksdb, leveldb

Why C++?

Posted by Edouard on 09 Feb 2021

Why C++?

Read More

Tags: c++, programming

Large scale Industrial IoT data project: lessons learned in 2020

Posted by Edouard on 08 Jan 2021

Introduction

Industrial Internet of Things (IIoT) is facing the new challenge of having to ingest very high volume of timeseries data and perform complex analysis in real time.

Read More

Tags: waveform, IIoT, timeseries, industrial, mqtt, predictive maintenance

Achieving maximum write speed with QuasarDB

Posted by Edouard on 22 Oct 2020

QuasarDB is the fastest timeseries database in the world by ingestion speed (and probably by querying speed, but it's very hard to establish objectively), by a significant margin. In the 3.9 branch, we furthered our advance further, and we are working to deliver another major performance boost in the following releases.

Read More

Tags: time series, performance, tuning

Three cool free extensions for VS 2019

Posted by Edouard on 19 Sep 2020

Although QuasarDB is mostly run on Linux, most of the development team works primarily on Windows using Visual Studio. It’s because VS has the most advanced native debugger, sitting on top of the Windows Debug API.

Read More

Tags: cpp, debugging, visual studio, extension

Improving our release process

Posted by Edouard on 27 May 2020

You might have noticed that we didn't release QuasarDB 3.9.0 on May 4th, 2020, as our previous release schedule would have commanded.

Since the beginning of the year, QuasarDB's user base grew faster than we anticipated, creating many pain points in our release process, especially the QA phase.

Read More

Tags: process, quasardb, release

auto abuse considered harmful

Posted by Edouard on 27 Apr 2020

Auto is a keyword introduced in C++ 11 out of the necessity to capture lambdas.

Read More

Tags: c++, auto

Using C++ containers efficiently

Posted by Edouard on 23 Mar 2020

Hello, dear reader! I think you have many assumptions about the performance and usage of data structures in C++. This blog post is me, destroying your world.

Read More

Tags: c++, performance, containers

QuasarDB on Kubernetes

Posted by Leon Mergen on 05 Mar 2020

Deploying QuasarDB in cloud-native environments has been possible for many years using the official Docker containers we provide. With the recent rise of container orchestration platforms such as Kubernetes, we can do much more: a single interface to provision, monitor and operate a QuasarDB cluster.

Read More

Timeseries: what are ASOF joins?

Posted by Edouard on 30 Jan 2020

In QuasarDB 3.5 we introduced ASOF joins. If you're new to timeseries databases, you may be confused about what ASOF joins are, and how useful they are.

Read More

Tags: quasardb, capital markets, asof

Ingesting 10 billion rows of timeseries data in 95 seconds

Posted by Edouard on 12 Dec 2019

Timeseries data = ingestion challenges

If there’s one thing that’s hard when managing timeseries data is just the sheer amount of it. Timeseries data has volume built-in because it’s cumulative. You don’t want a single picture: you want the whole movie!

Read More

Tags: quasardb, time series, performance

Introducing Delta4C: a high speed, adaptive, lossless compressor for timeseries

Posted by Edouard on 08 May 2019

This post is part of a series about the challenges behind database performance and how to accurately assess it.

Why compression matters so much for timeseries data

Whatever database engine you are using, efficient disk storage is always welcomed. When your 10 GiB become 100 GiB once in the database, that’s never a nice thing!

Read More

Tags: quasardb, time series, performance, compression, managed

Should you care about performance?

Posted by Edouard on 29 Apr 2019
This post is part of a series about the challenges behind database performance and how to accurately assess it.

You don’t care about performance

When we started selling QuasarDB, we focused on its performance advantages and touted how great they were. The logic behind that was obvious: we were very strong in this area; thus we should bring the battle to where we are strong!

Read More

Tags: quasardb, software, time series, performance

Benchmarking timeseries ingress

Posted by Edouard on 16 Apr 2019
This post is part of a series about the challenges behind database performance and how to accurately assess it.

Purpose of an ingress benchmark

When evaluating a timeseries database management system (later referred as TSDBMS or TSDB), one important dimension is the ingress speed (a.k.a. insertion or ingestion), that is, how fast the database can store new data points.

Read More

Tags: quasardb, time series, performance

Database performance

Posted by Edouard on 12 Mar 2019

This port is the first in a series about the challenges behind database performance and how to accurately assess it. In future posts, we will dig more into the specifics of benchmarks and design choices.

The Penrose stairs of performance

If you are following database innovation you can see that nearly every database vendor out there has at least one benchmark putting them in the first place. With every vendor selling the fastest database there is, you end up in a Penrose stairs situation where everyone is faster than everyone.

Read More

Tags: memory, optimization, process, software, performance

Secure by default

Posted by user on 18 Jul 2017

The security debacle. As a database user or administrator, you may have learnt that MongoDB recently took a very serious hit. A hit of over 28,000 hacked installs.

Read More

Tags: Security, quasardb, Uncategorized

Time series in quasardb

Posted by user on 10 Apr 2017

At first an interrogation - At the beginning, there was a question. Why are we waiting for machines to give us an answer? Why aren't we able to analyze everything in real time?

Read More

Tags: analytics, big data, quasardb, time series, Uncategorized

C++ Braced Initialization

Posted by user on 05 Mar 2017

Since C++ 11 it's possible to use braces for construction and initialization. Although this is something you could ignore for the code you write, it's obviously important to know for the code you may read.

Read More

Tags: cpp, braces, initialization, Uncategorized

When noexcept?

Posted by user on 12 Dec 2016

In a previous post, we had a look at the new constexpr keyword that has been introduced in C++ 11. Today we'll study another new fancy specifier: noexcept.

Read More

Tags: cpp, noexcept, Uncategorized

Demystifying constexpr

Posted by user on 22 Nov 2016

C++ 11 and C++ 14 came with a lot of new features. People tend to focus on lambdas and rvalue references, but today I’d like to talk about constexpr.

Read More

Tags: c++, constexpr, Uncategorized

Forward and backward compatibility in quasardb

Posted by user on 04 Jul 2016

I think the number one request from all our existing customers is to have as much as backward compatibility as possible with quasardb.

Read More

Tags: compatibility, quasardb, Uncategorized

Right to data

Posted by user on 04 May 2016

Computers have been invented to compute large amounts of data at very high speed, relative to the human brain speed. In a sense, computers have always been about "big data". Although "big data" is probably more a marketing term than a technical one, we must acknowledge that we are at a threshold where data mining can be done on such amounts that new approaches are emerging.

Read More

Tags: activepivot, big memory, hadoop, nosql, quartetfs, quasardb, spark, Uncategorized

C++ complexity: it's a feature

Posted by user on 22 Jan 2016

One of the constant reproaches we can hear about C++ is that it is too complex, too hard, too unsafe and has too many features, really these r-value references are useless and dangerous and why would you need variadic templates anyway, and "lol" you are still managing memory manually in 2016.

Read More

Tags: cpp, programming, rant, Uncategorized

Debugging in GDB from Visual Studio

Posted by user on 01 Dec 2015

I wanted to share some tips and tricks for those of you who use a recently released Visual Studio GDB Debugger extension. You may want to read the announcing blog post as well.

Read More

Tags: debugging, gdb, Uncategorized, visual studio

Weird PIE-related linker errors on OS X

Posted by user on 20 Nov 2015

Weird error - Recently, I have encountered a really strange linker error after a ridiculously small change in the code. Look at the error message and find the reason in 5 seconds without looking further in the text and I owe you a beer ;).

Read More

Tags: linker, OS X, Uncategorized, visibility

Finding and fixing leaks in our nodejs addon

Posted by user on 02 Nov 2015

nodejs is a great platform reaching a certain degree of maturity. We generally have a lazy approach toward API support with quasardb: if no one is asking for it, don't implement it.

Read More

Tags: cpp, addon, nodejs, quasardb, Uncategorized

Building a database

Posted by user on 05 Oct 2015

It is with great pleasure and pride we announce the fifth beta of our quasardb 2.0 release. Things are getting exciting as we are approaching the final release!

Read More

Tags: history, nosql, quasardb, release, Uncategorized

Automatically mounting instance stores on an AWS AMI

Posted by user on 13 Aug 2015

On Amazon's EC2, using EBS as the backend storage for your application has been the de-facto standard. Using the local storage of an EC2 container is risky: data loss occurs when a container is stopped and it is not replicated by default. As such, people should default to using EBS, which is Amazon's version of a SAN.

Read More

Tags: administration, amazon, aws, quasardb, Uncategorized

5 awesome C++ libraries we use

Posted by user on 20 Apr 2015

This is an opinionated post about five libraries we use in the production code of quasardb.

Read More

Tags: Uncategorized

SFINAE Hell: detecting template methods

Posted by user on 12 Apr 2015

One of the cool thing about C++ is all these checks you can do at compile time. With compile time checks you can not only write safer code, but also faster code for the simple reason that alls the checks you do at compile time will not have to be done at runtime.

Read More

Tags: Uncategorized

Chaotic testing

Posted by user on 22 Mar 2015

The stabilization algorithm - quasardb is a distributed, peer-to-peer database based on the Chord Algorithm. It is actually much more than that, but for the purpose of this post we will focus on the Chord algorithm and one fundamental process: stabilization.

Read More

Tags: quasardb, testing, Uncategorized

quasardb on Microsoft Azure

Posted by user on 06 Feb 2015

Getting your hands on quasardb has never been that easy. With just a few mouse clicks you can deploy your own quasardb server in Microsoft Azure.

Read More

Tags: azure, quasardb, Uncategorized

Boost.Filesystem and error handling

Posted by user on 04 Feb 2015

Many Boost.Filesystem functions and methods (and the pattern also exists in Boost.Asio) will throw a boost::system::error_code in case of error, and sometimes it is not as obvious as you might expect, case in point:

Read More

Tags: boost, c++, error, exceptions, tie, Uncategorized

std::chrono ambiguities

Posted by user on 30 Jan 2015

Precision is costly - As you saw in our previous post getting a timestamp as precise as possible is expensive in precious cycles. We do it only when we really need it because we really care about writing the world fastest database.

Read More

Tags: c++, chrono, libc++, stl, Uncategorized, visual studio

This blog is moving

Posted by user on 22 Dec 2014

This is the last year for Bureau 14 as we unanimously decided to rename the company to quasardb.

Read More

Tags: Uncategorized

Include what you use

Posted by user on 17 Jan 2014

Oh what a tangled web we weave when we first try to optimize files inclusion in C++. I can hear your sarcasm other-language programmer developer, feel free to flood the comments sections with something along the lines of “We don’t have this problem in fancy-new-language, you paleo-programmer!”.

Read More

Tags: cpp, boost, clang, Uncategorized

Season’s greetings

Posted by user on 21 Dec 2013

I wish you all a great holiday season, see you next year!

Read More

Tags: holiday, Uncategorized

How to know the duration of a software project

Posted by user on 30 Nov 2013

The best way to have a value as accurate as possible is to complete the software project.

Read More

Tags: Uncategorized

The Nagle Algorithm

Posted by user on 25 Nov 2013

I'm used to read technical inaccuracies when it comes to system programming, and writings about the Nagle algorithm are very high in the list.

Read More

Tags: asio, c++, nagle, networking, Uncategorized

On the importance of automated benchmarks

Posted by user on 11 Sep 2013

A couple of years ago, Antirez wrote a note on his blog about "The Toilet test". He made a couple of great points, criticizing the test and the method.

Read More

Tags: Uncategorized

How fuzzing helped us find a nasty bug

Posted by user on 08 Jul 2013

> The issue described in this post has been fixed in quasardb 1.0.1 released today. We strongly encourage our customers to upgrade their clusters. This issue can cause a denial of service if an attacker has access to the network where your quasardb cluster is installed.

Read More

Tags: c++, fuzzing, quasardb, testing, Uncategorized

Scaling with C++ 11

Posted by user on 21 Jun 2013

My "Scaling with C++ 11" talk is online.

It's really great to see yourself in video because everything bad about your presentation becomes obvious. I already have a lot of ideas to improve this one and I hope I can deliver something much better at Meeting C++. I hope to see you there!

Read More

Tags: boost, c++, conference, Uncategorized

Lockheed SR-71 Blackbird

Posted by user on 12 Jun 2013

In my C++ Now 2013 presentation, I talked about the Lockheed SR-71 Blackbird as an example of how you sometimes go around problems by "leaking":

Read More

Tags: blackbird, c++, sr-71, Uncategorized

The Ax Thief

Posted by user on 19 May 2013

As I flew back to France, I was reading Lie Yukou:

Read More

Tags: liezi, tao, Uncategorized

String to int and vice versa

Posted by user on 15 May 2013

> Hope you guys are enjoying C++ Now!

We use Boost.Spirit for all our parsing and generation. What I mean by that is that when you have to parse input or generate output, we don’t use std::stringstream, we don’t use the C libraries and we definitely don’t write custom parsers or generators.

Read More

Tags: boost, c++, conversion, spirit, Uncategorized

The Bureau 14 coding rules

Posted by user on 22 Apr 2013

We don't actually have "coding rules" at Bureau 14, at least, not in the sense you would expect them.

One one hand, we think it's best to give as much freedom as possible to the developers and prevent rules to get in the way of evolution.

Read More

Tags: bureau 14, c++, coding, rules, Uncategorized

Seasons greetings

Posted by user on 27 Dec 2012

"Am I indeed possessed of knowledge? I am not knowing. But if a mean person, who appears quite empty-like, ask anything of me, I set it forth from one end to the other, and exhaust it."
--Confucius

We wish you all happy hollidays!

Read More

Tags: Uncategorized

Function composition in C++11

Posted by user on 19 Nov 2012

There is a lot of code in this post, it will probably not compile "as is" because of missing headers or compiler's incompatibilities. Nevertheless, we made available through our open source initiative the full working library on github. It has been tested on Clang 3.1, gcc 4.6.1 and MSVC 11 on FreeBSD, Linux and Windows.

Read More

Tags: boost, c++, functional, quasardb, Uncategorized

The Butterfly Bug

Posted by user on 26 Sep 2012

At Bureau 14 we highly value code quality: it's the corner stone of our software development strategy and certainly one of our core values as a company. We believe that investing in code quality pays off in the long term, especially when our software is entrusted by our customer with mission-critical data.

Read More

Tags: bug, butterfly, driver, heisenbug, kernel, Uncategorized

For the sake of productivity...

Posted by user on 10 Sep 2012

By judging the endless stream of blog entries, books, conferences, tweets, status updates, advices, statements and aphorism on the topic, you can certainly figure out that productivity is the hottest topic behind celebrities naked pictures, cats in embarrassing situations and tomorrow’s weather.

Read More

Tags: method, productivity, sleep, Uncategorized

NoSQL is a term doing us a disservice

Posted by user on 01 Feb 2012

Although we edit NoSQL software, I always disliked calling our product a NoSQL database. I know we say in the third paragraph of our web page that wrpme is a NoSQL database, but see, right after it says that we prefer to call it a postmodern database as proposed by Dr. Richard Hipp.

Read More

Tags: acid, altsql, database, nosql, rdbms, transaction, Uncategorized, wrpme

Parrots

Posted by user on 05 Nov 2011

Parrots are wonderful creatures of Earth. How many animals can you actually talk to?

Read More

Tags: parrot, pet, pirate, Uncategorized

On quality and code size

Posted by user on 26 Sep 2011

I recently came across a great blog post asserting - based on numerous studies - that size is the best predictor of code quality.

Read More

Tags: agile, process, programming, quality, Uncategorized, unit testing

Setting up a high performance wordpress server

Posted by user on 18 Apr 2010

Our website isn't critical, in the sense that, if the website goes down, our operations remain unaffected. However, lately, the website was becoming extremely slow and it was turning into bad publicity.

Read More

Tags: freebsd, memcached, mysql, nginx, Uncategorized, wordpress

BoostCon 2010

Posted by user on 16 Mar 2010

The BoostCon is a Boost centric conference, but it's actually much more than that. It's probably the best C++ conference in the world.

Read More

Tags: boost, boostcon, c++, Uncategorized

Security through deception

Posted by user on 03 Jan 2010

In battle, use direct forces to match the enemy and use indirect forces to win the enemy.-Sun Tzu

Read More

Tags: Security, copy protection, reverse engineering, Uncategorized

A thing that really improves code quality

Posted by user on 23 Dec 2009

Why increase quality? One good reason: costs reduction. There are many others, but this one is important enough.

Read More

Tags: programming, projects, quality, Uncategorized, unit testing

You can also do that kind of things in Haskell

Posted by user on 18 Dec 2009

Following up on the discussion about programming languages, I wanted to share with you a demo made in Haskell (demos are generally written in C, C++ or even assembly language).

Read More

Tags: demo, haskell, Uncategorized

Smart pointers are overused

Posted by user on 31 Aug 2009

Smart pointers have become the premium choice for memory management in C++. This is everything but odd, when you come to think of it.

Read More

Tags: boost, c++, memory, smart pointers, stl, Uncategorized

Unit tests: the obvious, the bad and the good

Posted by user on 20 Aug 2009

A while ago I technically screened a developer for one of our biggest client. This developer was a huge fan of unit testing. Whatever software puzzle I submitted could be solved with an unit test.

Read More

Tags: agile, Uncategorized, unit testing

The ultimate secret of successful projects

Posted by user on 28 Jul 2009

I'm obsessed with theorizing everything in life. That can be a good thing, and that can be a bad thing. When you know how to stop, I found it can be quite a strength.

Read More

Tags: agile, lean, management, organization, recruitment, scrum, toyota, Uncategorized

What is heap fragmentation ?

Posted by user on 05 Jul 2009

Let me tell you why you're here. You're here because you know something. What you know you can't explain, but you feel it. You've felt it your entire life, that there's something wrong with the world. You don't know what it is, but it's there, like a splinter in your mind, driving you mad. It is this feeling that has brought you to me. You want to know what is heap fragmentation.

Read More

Tags: boost, c++, heap, Uncategorized, virtual memory

Reducing heap pressure

Posted by user on 14 Jun 2009

C++ is the world of do-it-yourself memory management. When discussing programming with someone, as I reach the point where I tell that I mainly work with C++ the first thing I hear is "Oh... But you have to deallocate memory yourself... That's so cumbersome and obsolete!"

Read More

Tags: boost, c++, heap, pool, Uncategorized

Securing a software company - Part I

Posted by user on 21 May 2009

I was working on a typical "airport-security-I-know-better-post", then I realized that Bruce Schneier already said all of these things better than me.

Read More

Tags: Security, network, social engineering, Uncategorized

7 + 7

Posted by user on 12 May 2009

While still a bit jetlagged, I managed to find the time to setup my new box, a Core i7 running Windows 7 64-bit. A 7/7 box, which in itself is pretty neat.

Read More

Tags: Core i7, SSD, Uncategorized, Windows 7

What we really need for debugging

Posted by user on 10 May 2009

I just returned from the BoostCon 2009, expect some new content very soon!

Read More

Tags: aspen, psychic, Uncategorized

UTF-8 strings and Visual C++

Posted by user on 30 Apr 2009

MAnd Unicode came to this world - Most of us don't work in ANSI or ASCII anymore. Most of the projects I work on are either UTF-8 or UTF-16.

Read More

Tags: c++, Uncategorized, unicode, visual studio

Hardware is expensive

Posted by user on 28 Apr 2009

You probably heard at least once the hardware is free aphorism. Perhaps you even said it! The underlying notion is that optimizing code is generally more expensive than upgrading the machine or even buying a new one.

Read More

Tags: optimization, programming, software, Uncategorized

Why I want to play with Haskell

Posted by user on 27 Apr 2009

Haskell, a sort of LISP. The language with parenthesis. Lots of parenthesis. LISP has got lots of parenthesis, not Haskell. Not to say that there aren't parenthesis in Haskell.

Read More

Tags: c++, haskell, Uncategorized

BoostCon 2009

Posted by user on 25 Apr 2009

I will attend BoostCon 2009. I hope to see you there!

Read More

Tags: boost, Uncategorized