Chart Notes

Chart Notes

In the case of these Database Benchmark charts there are several factors to consider, such as

• These graphs show the database benchmarks submitted to us from users of our PerformanceTest V11 software.
• Baselines can be submitted from anywhere therefore these are global statistics.
• The segment of users of PerformanceTest may be geared toward system builders and enthusiast.

Good:

• Measure the relative performance of different hardware.
• Measure the impact of client server latency (local vs remote databases).
• Compare performance of different databases on the same hardware.
• Compare single threaded vs multi-threaded DB behaviour.

Limitations:

• Only basis CRUD functionality is measurement using a set of features common to all SQL based databases. Advanced functions are not tested.
• Tests are limited to SQL style databases.
• Cloud instances are not always the same hardware, even if the name of the instance is the same. Hardware configurations can also vary over time.
• As with real databases client server physical separation distance (latency) has a huge impact on performance. Even when the same hardware is used.
• Databases are very configurable. Different engines, cache levels, etc.. No attempt was made to tune the databases for maximal performance.

Database Test Descriptions

Database Rating:

Advanced Database Test measures the throughput and latency of the CRUD operations by performing two test runs, first a single-thread and the second run with multi-threaded (50 threads) test.
Database Rating is calculated from the throughput results of the two test runs using weighted scoring method with the percentages as follows:

• Run1 - Single INSERT: 10%
• Run1 - Bulk INSERT: 10%
• Run1 - SELECT: 13%
• Run1 - UPDATE: 12%
• Run1 - DELETE: 5%
• Run2 - Single INSERT: 10%
• Run2 - Bulk INSERT: 10%
• Run2 - SELECT: 13%
• Run2 - UPDATE: 12%
• Run2 - DELETE: 5%

Insert (Single):

The Single Insert test executes insertion one row in a single SQL statement. There are four database tables and it will insert 200 records into each table, in total 800 rows will be inserted using single insertion SQL query.
In multi-threaded test, the same total number of rows will be inserted, and it is done by dividing the tasks equally to each thread and inserting simultaneously into tables.

Insert (Bulk):

The Bulk Insert test inserts 100 rows in a single SQL statement.

Select:

Select is most common and important database operation and we have assigned a higher weight (13%) to Select than others.
We have designed six different Select query statement for an effective database testing, covering functions and operators like COUNT(), SUM(), GROUP BY, ORDER BY, JOIN, LIKE.

Update:

Update is also an important command commonly used in everyday database operations. A higher weight (12%) is assigned to this and there are two different queries executed for the Update test.

Delete:

The Delete test will delete 10,000 rows in total from a selected database table.

How to Add Your Own Database Benchmark

• Download and install PerformanceTest V11 or later.
• Start PerformanceTest then from the Advanced menu bar select "Advanced Database Benchmark...".
• Select the Standard Test test type, the desired Database to run on (and if required, fill in the required fields for the server connection).
• Once the tests have completed, select "Upload Baseline to Web".

Your results (if qualified) will not appear in the graphs immediately as the graphs are only updated once per day. Additionally, some systems may be filtered out where there is suspicion of cheating or inappropriate results.