Monday, April 11, 2011

MySQL Cluster doing 6.82M reads per second

We ran a number of tests to see how many reads per second we could get from MySQL Cluster. We used a modified version of flexAsynch (as shown in previous blog), where each record read was 100 bytes in size.

With a cluster of 4 data nodes operating on 2 machines we were able to process 1.15M reads per second. On a cluster consisting of 8 data nodes executing on 4 machines we were able to process 2.13M reads per second. On a 16-data node cluster with 8 machines used for data nodes, we were able to process 4.33M reads per second and finally a cluster with 32 data nodes distributed on 16 machines we executed 6.82M reads per second. The tests were run on MySQL Cluster 7.1, we're confident that similar numbers can be achieved with MySQL Cluster 7.0 and also with the new beta version MySQL Cluster 7.2.

This benchmark will give you a good idea what can be achieved with direct usage of the NDB API, and using other APIs like Cluster/J, mod-ndb, NDB-memcached.
at

6 comments:

  1. Anonymous9:38 AM

    Hi Mikael,

    MySQL 7 is supposed to scale with multi core processor, but for the benchmarks you are using several data nodes per machine; so I guess data nodes does not scale properly with several cores. Could you comment something on this?

    Thanks in advance,

    Evaristo Camarero

    ReplyDelete
  2. Mikael Ronstrom10:49 AM

    MySQL Cluster data nodes scale to 6-8 cores/threads. The machines I used had 12 cores/24 threads. Thus I could easily house 2 data nodes per machine.

    ReplyDelete
  3. Darpan Dinker11:16 PM

    Mikael,

    The numbers in this blog look impressive.

    (1) It will be great to provide the benchmark or its detailed definition.

    (2) What was the overall use of machines to generate the load

    (3) What are the results of flexSync?

    Thanks,
    Darpan

    ReplyDelete
  4. Mikael Ronstrom6:41 PM

    The benchmark was run using the tool that I wrote about in my blog today together with an updated version of the flexAsynch which hopefully will soon be a part of a release.

    Each data node used up about 5.5 CPU cores. The 6.82M reads per second used a cluster of 32 data nodes.

    I haven't tested flexSync (it's called flexBench and was the original benchmark program for MySQL Cluster, not used so much anymore).

    ReplyDelete
  5. Anonymous8:09 PM

    Hi,

    We are using version 7 and we cant get anywher enear these numbers. We managed to get about 4000 selects per secodn using 2 machines 2 nodes and 32 threads. Can you publish your config files etc so we can see what you are doing. We used Intel i5 2.8 GHZ machines for our tests

    ReplyDelete
  6. Mikael Ronstrom9:46 PM

    Hi,
    The benchmark is flexAsynch, the correct version to use for this can at least be found in the LaunchPad tree of MySQL Cluster 7.x, not sure exactly which version it came into.

    The benchmark program I used to drive the benchmark can be found at http://dev.mysql.com/downloads/benchmarks.html

    There is a lot of documentation in the tarball of how to setup and run the benchmark.

    The most important thing to consider in the configuration is to ensure to set MaxNoOfExecutionThreads to 7 or 8. Also to lock to CPUs so that each thread has its own CPU (there is a config variable for this).

    ReplyDelete