We have worked hard on ensuring stability and adding the required features for our customers lately. Thus the RonDB 21.04.15 release has reached a very high quality level and will be able to sustain users of it until they desire to upgrade to a newer release of RonDB.
Most of the changes in this release is related to the new REST API server that makes it possible to read using single reads or batch reads using primary key lookups through a REST protocol or through a gRPC protocol. The REST API server also supports reading directly from the Hopsworks Feature Store that takes into account the metadata model of the Hopsworks Feature Store.
Much of the work around RonDB is centered around automated management of RonDB. To this end we have developed the ndb-agent that makes it possible to create a cluster, stop the cluster, start the cluster again, take a backup, delete a backup, restore from backup and finally to reconfigure the cluster as an online operation.
Reconfigure the cluster means adding or removing replicas, increasing the size of data node VMs. It means that MySQL Server VMs can be added, changed and dropped as needed by the application.
All of those operations are already operational and working. We are now working on an improvement that speeds up the change process significantly. Adding a new MySQL Server can now be done in 2-3 minutes and most of this time is spent on creating the new VM in the choosen cloud (Hopsworks supports AWS, GCP and Azure).
The new ndb-agent works in the same fashion as Kubernetes through maintaining a desired state. This means that it is fairly straightforward for the ndb-agent to support both our cloud offering and a Kubernetes setup.
RonDB development is now focused on the new RonDB release 22.10.1. This will introduce 8 new features. The most important feature is supporting variable sized disk columns. RonDB 22.10 has been in development and testing for almost 3 years already, so it is already a very stable release. It brings in addition a number of performance improvements.
The release notes for RonDB 21.04.15.
The full set of new features in RonDB 21.04.
The full set of new features in RonDB 22.10.
The new Hopsworks release also makes use of Replication between RonDB clusters. A Hopsworks cluster can use a single small RonDB cluster and can grow into an Enterprise setup with several large RonDB clusters and replicated between regions far away from each other.
RonDB is used to handle the Online Feature Store, the metadata of the Hopsworks Feature Store and the metadata of HopsFS. HopsFS is the storage of the Offline Feature Store. HopsFS is a distributed file system that can store many petabytes of data in an efficient manner. Hopsworks Offline Feature Store makes use of DuckDB to perform complex analysis of the data to train AI models and perform batch inferencing.
Thus RonDB is a critically important component in the next generation AI system developed at Hopsworks. All large companies around the world is considering how they can build their AI models and supporting system. Hopsworks is providing a platform for those companies, both small and very large companies.
Hopsworks provides a free service where anyone can get a free Hopsworks account at https://app.hopsworks.ai and try out the service themselves.
