# Large cluster perf #1 - 25 nodes

**URL:** <https://forum.yugabyte.com/t/large-cluster-perf-1-25-nodes/58>\
**Category:** General\
**Created:** [October 20, 2017, 8:13pm UTC](https://forum.yugabyte.com/t/large-cluster-perf-1-25-nodes/58 "2017-10-20T20:13:52Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![karthik](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.yugabyte.com/karthik/32/13_2.png) [@karthik](https://forum.yugabyte.com/u/karthik)\
**Post date:** [October 20, 2017, 8:13pm UTC](https://forum.yugabyte.com/t/large-cluster-perf-1-25-nodes/58/1 "2017-10-20T20:13:52Z")

</div>

We are currently doing performance benchmarking with large YugaByte cluster, and are excited to share the results of the first benchmark in this series.

## Setup

Here is the benchmark setup:

- 25 nodes in Google Compute (GCP)
- Each node is a n1-standard-16
  - 16 vcpu’s
  - Intel(R) Xeon(R) CPU @ 2.20GHz CPUs
  - 60GB RAM
  - 2 x 375 GB direct attached SSD

- Replication factor = 3
- YugaByte Cassandra key-value workload
  - 40 byte keys
  - 16 byte values

You can find the source code for the key-value application [here](https://github.com/yugabyte/yb-sample-apps/blob/master/src/main/java/com/yugabyte/sample/apps/CassandraKeyValue.java), and some documentation on [developing apps on YugaByte](https://docs.yugabyte.com/preview/develop/).

## 100% Reads

- 1.3M read ops/sec
- Around 0.3ms latency on the server side
- 65% average CPU on the YugaByte nodes

## 100% Writes

- 500K write iops/sec
- Around 1.5ms latency on the server side
- 67% average CPU on the YugaByte nodes

We are able to see a linear scale-out as we go from 3 nodes all the way to 25 nodes. Stay tuned for results with larger cluster sizes as well as YCSB benchmarks!

---

<div class="post-metadata">

**Author:** ![ddorian43](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.yugabyte.com/ddorian43/32/60_2.png) [@ddorian43](https://forum.yugabyte.com/u/ddorian43)\
**Post date:** [January 19, 2018, 7:51pm UTC](https://forum.yugabyte.com/t/large-cluster-perf-1-25-nodes/58/2 "2018-01-19T19:51:12Z")

</div>

If you guys can benchmark with in-memory & against scylladb too that would be great.

---

<div class="post-metadata">

**Author:** ![karthik](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.yugabyte.com/karthik/32/13_2.png) [@karthik](https://forum.yugabyte.com/u/karthik)\
**Post date:** [January 22, 2018, 4:50am UTC](https://forum.yugabyte.com/t/large-cluster-perf-1-25-nodes/58/3 "2018-01-22T04:50:54Z")

</div>

Will do @ddorian43 - give us some time, but we’ll get to it soon.

By in-memory - did you mean in-memory tables?

---

<div class="post-metadata">

**Author:** ![ddorian43](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.yugabyte.com/ddorian43/32/60_2.png) [@ddorian43](https://forum.yugabyte.com/u/ddorian43)\
**Post date:** [January 22, 2018, 10:19am UTC](https://forum.yugabyte.com/t/large-cluster-perf-1-25-nodes/58/4 "2018-01-22T10:19:10Z")

</div>

Meaning data smaller than memory. So after warmup, you’re testing the caching layer when doing read-requests only.

---

<div class="post-metadata">

**Author:** ![harsha549](https://avatars.discourse-cdn.com/v4/letter/h/74df32/32.png) [@harsha549](https://forum.yugabyte.com/u/harsha549)\
**Post date:** [April 21, 2020, 10:41am UTC](https://forum.yugabyte.com/t/large-cluster-perf-1-25-nodes/58/5 "2020-04-21T10:41:08Z")

</div>

I was curious with @ddorian43 question.Did YB do a benchmark ?

---

<div class="post-metadata">

**Author:** ![karthik](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.yugabyte.com/karthik/32/13_2.png) [@karthik](https://forum.yugabyte.com/u/karthik)\
**Post date:** [April 21, 2020, 6:46pm UTC](https://forum.yugabyte.com/t/large-cluster-perf-1-25-nodes/58/6 "2020-04-21T18:46:14Z")

</div>

Hi @harsha549,

Yes, we did try this out and actually ascertain that we can serve sub-millisecond latencies on a public cloud. Here is post detailing this (this is an older post, our perf is much better compared to this older version): [Achieving Sub-ms Latencies on Large Datasets in Public Clouds | Yugabyte](https://blog.yugabyte.com/achieving-sub-ms-latencies-on-large-data-sets-in-public-clouds/)

Summary:

> We loaded about 1.4TB across 4 nodes, and configured the block cache on each node to be only 7.5GB. **On four 8-cpu machines, we were able to get about 77K ops/second with average read latencies of 0.88 ms.**