Flowdesk - using in-memory database to make digital finance fast and accessible

The use of an in-memory database platform on Google Cloud is helping French firm Flowdesk deliver on its company mission of bringing the best of crypto finance to clients on a global basis.

The software - Redis Enterprise - is enabling the execution of more than 1 million orders per day with sub-millisecond latency and zero downtime, says its Head of Infrastructure, Julien Tocci.

Headquartered in Paris, Flowdesk markets a cryptocurrency trading platform. Tocci, who’s based out of its Singapore office, heads up the company’s DevOps, cloud, data, and cybersecurity operations.

That’s far from a trivial job, as Flowdesk’s core proposition relies on the intensive use of its infrastructure, given its commitment to algorithmic trading and crypto.

It applies these services to a form of financial instrument called market making, where a customer is offered both a buy and a sell price in a tradable asset.

In normal finance, brokerage houses are the most common types of market makers.

Tocci says:

We are a market maker, but we do things differently from the classical market makers; we provide a tech-based service, so market making as a service. That’s because we believe that’s way more transparent and capital efficient for clients and overall, and is just a better way to do it.

From that departure point, says Tocci, the firm developed a trading platform that connects 100-plus centralized and decentralized exchanges.

Some of the client services a crypto user can buy off the platform include a range of financial tools beyond market making, such as custody, brokerage, and treasury management. 

Dynamic configuration management

Before it opened its virtual doors for business in 2020, this cloud native company started off using a combination of Cockroach and Google Cloud Memorystore, says Tocci.

Initially, that did allow sub-second access to the order books that store financial data, he says, and met the brand’s need for a high-availability, low-maintenance database service that worked well with its chosen hyperscaler platform, Google Cloud.

Tocci explains why this approach to working with data in the cloud was chosen:

We have two kinds of data: live data, so transaction data that is going very fast and is time sensitive. Then, we have all the data related to those transactions we also want to store - so a full order book will be in the cache, and every update of this order book will be the live data. From the start, we knew we’d always have a lot of data that we wanted to cache.

But as the company started to grow, he says, it soon started to see a need for dynamic configuration management—i.e., to be able to better structure the configuration of its compute workloads.

As the firm started to operate in both Europe, the US, and Far East, this was soon joined by a need for global availability and a multi-regional cloud architecture, he states.

While GCP is still Flowdesk’s dominant cloud home, 5% is now also in Microsoft Azure.

Tocci decided that to scale his database infrastructure, database-as-a-service would better suit these multiple needs than what he’d started with.

He says he was originally not aware of his final in-memory data partner but decided to evaluate the technology after some initial research.

Several vendors were looked at and several benchmarks were conducted, he says.

Redis eventually won out for a number of reasons, both operational and functional.

For one thing, the service is fully integrated with Google Cloud Marketplace, so he’d only have to pay one bill instead of two.

Flowdesk did originally try to build the integration using the on-prem version of Redis Enterprise but found it too hard to maintain - it was much easier to run as a cloud service, he states.

Tocci also liked that he can interact with the application within a private network using a technique called Virtual Private Cloud (VPC) peering, where resources in each network can communicate with each other fast and securely.

The team also likes all the product’s cloud interfaces, which he says are very good - but it’s the flexibility that seals the deal. In his words, Tocci says:

It’s very easy for us to scale up or scale down--when we have periods of high market activity, we can easily scale up the databases so they can meet the load. But equally when nothing much is happening in our market, we can reduce the database’s size, which is really cost efficient for us.

Engineering the financial institutions of tomorrow

Now that database-as-a-service has been at the core of Flowdesk’s workflow for two years, what does Tocci see as its ongoing contribution? He says: 

The first big thing that this permitted us to do was multi-cloud, so we now have a fully geographically distributed, database-replicated system. This means we have the database locally in three regions of the world, so we have very low latency, as all the data is so near. 

It’s also very easy for us to replicate keys between the regions, so there’s not a lot of operational complexity—it’s all very plug and play. With other database options there’s a tendency to overscale, which costs.

The second main benefit from this approach to data caching, he says, is fast scaling—at peak times, the platform can handle 100,000 operations per second.

He also reports zero operational incidents in the time the system has been live: uptime has been “very good.”

Next steps for Flowdesk and cloud caching will be adding new regions to trade in.

He also wants to improve efficiency, looking at every kind of latency within the system at every level, including at the data-handling level.

He concludes:

Scalability that results in better crypto markets is our focus, as we want to get to 1000 token issuers in the future. And with Redis and Google Cloud, we think we can really do that.