In MySQL Cluster 8.0.20 a new feature was added that enables higher write rates
in NDB Cluster. In previous releases the communication between nodes in the same
node group was handled by one socket. This single socket was able to handle about
200-300 MBytes of write traffic between the replicas inside one node group. Thus
in clusters with many node groups the write rates can be much higher.
For most applications this doesn't represent any problem at all. However there are
certain applications that require wast amount of writes.
One such application is when NDB is used to store file data. File writes can easily
go beyond 200-300 MBytes per second. Another application is when NDB is used to
store data from measurements, stock data or other applications that require millions of
writes per second.
The solution to this problem is to simply use multiple sockets when communicating
between nodes in the same cluster. This removes the bottleneck on write rates
between nodes in the same node group. The solution is applied by default since we
haven't seen any negative effects of using it. The number of sockets will be
derived from the number of LDM threads used in the data nodes.
We have verified the feature through the use of YCSB where the data is stored in
a disk data column. This is a good representation of writes of large amounts of
file data to NDB.
The second type of verification is through DBT2, an open source variant of TPC-C
where each benchmark thread starts a new operation immediately after completing
the previous transaction. This benchmark was previously limited by the single
socket such that scaling beyond 6 LDM threads didn't improve numbers at all.
With this change we can see scaling to full bare metal servers in the Oracle Cloud
with 52 CPU cores.
Reports of those benchmarks will be in separate blogs.