YugaByte DB P99 latencies with Netflix Data Store Benchmark

General
1

Recently we shared YCSB benchmark results for YugaByte DB. Apart from YCSB, Netflix Data Store Benchmark (NDBench) is another publicly available, cloud-enabled benchmark tool for data store systems. We ran NDBench against YugaByte DB for 7 days and want to share our results:

Setup

YugaByte DB machines:

  • 6-node cluster in Amazon Web Service (AWS). Each node is a i3.2xlarge machine with:
    • 8 vCPUs Intel® Xeon® CPU E5-2686 v4 (Broadwell) @2.3 GHz
    • 61GB RAM
    • 1.9TB direct attached SSD
    • Same availability zone
  • Replication factor = 3
  • Default YugaByte DB configuration parameters were used.

NDBench machine:

  • 1 c3.xlarge machine
    • 4 vCPUs Intel® Xeon® CPU E5-2680 v4 (Ivy Bridge)
    • 7.5GB RAM
  • Same availability zone as YugaByte DB cluster.

NDBench configuration used:

# Data size
ndbench.config.numKeys=20000000 # Backfill keys
ndbench.config.numValues=1000
ndbench.config.dataSize=1024
ndbench.config.cass.colsPerRow=8
# Number of worker threads
ndbench.config.numReaders=8
ndbench.config.numWriters=8
# Consistency level
ndbench.config.cass.writeConsistencyLevel=LOCAL_QUORUM
ndbench.config.cass.readConsistencyLevel=LOCAL_QUORUM
# Request per second (RPS)
ndbench.config.readRateLimit=7200
ndbench.config.writeRateLimit=1800

Results

ndbench
ndbench751×597 35.2 KB

  • Execution time: 7 days (163 hours)
  • Total successful reads = 4.11 billion, writes = 1.05 billion
  • Latencies (milliseconds) observed:
    • Read: Avg = 1.368ms, P95 = 3.311ms, P99 = 5.722ms, P995 = 6.866ms
    • Write: Avg = 1.803ms, P95 = 2.299ms, P99 = 3.311ms, P995 = 4.768ms

Throughout the entire run, compactions were enabled and running in the background. In spite of that, our P99 (99% percentile) read/write latencies were still under 6 ms. These contrast significantly to much higher P99s seen with Apache Cassandra - primarily due to architectural reasons (eventual consistency needs background anti-entropy/read-repairs; quorum reads to achieve higher consistency put multiple servers in the critical path of each request) as well as implementation choices (Java implementation, GC pauses, etc.).

2 Likes