In MySQL Cluster 8.0.20 we made it possible to write to disk data columns
at considerably higher write rates than before. In MySQL Cluster 7.6 we
introduced adaptive checkpointing of in-memory rows to ensure that restarts
can be faster.
In order to handle very high write rates (up to more than 1 GByte per second)
on disk data columns in NDB it is necessary to control write rates more
To get the most efficient use of disk bandwidth it is important to ensure that
write rates are kept fairly constant such that we constantly make use of the
disk bandwidth. If the load on the disks goes up and down too fast we will not
be able to make use of all the available bandwidth in the disks.
With modern NVMe devices found in e.g. the Oracle Cloud one can achieve 10-20
GByte of reads and writes to a set of 8 NVMe devices using specialised disk
benchmarks. Achieving this with a DBMS is not as easy, but we were able to
see NDB use 7 GBytes per second of reads and writes in some YCSB benchmarks.
This represents one more step in the direction of adaptive algorithms in NDB
Cluster to ensure that users always will get the optimal behaviour of their
hardware when using NDB.
Some previous steps are:
- Adaptive send assistance
This feature was introduced in 7.5 that made it possible for all threads
to assist the send threads to send messages on sockets. The main input to
the adaptive algorithm is the amount of CPU time spent in each thread for
normal execution, sleeping, assisting send threads.
- Adaptive in-memory checkpoint speed
This feature adapts the speed of checkpoints based on the amount of write
activity in NDB Cluster.
- Adaptive CPU spinning
This is introduced in 8.0.20 and will be presented in separate blog.