Deployment in two data centers

General
1

Hi, please help me figure this out.

I need to deploy a multi data center cluster. Exactly 2 data centers. Two clusters with bidirectional replication are not suitable, since we have many tables that need to be replicated. And in the setup_universe_replication command you need to specify the IDs of all tables. It seems there is no way to specify that certain databases need to be replicated.

Would it be ok to do the following setup instead? 3 nodes in one data center and 3 nodes in another data center in one cluster. On each node there is a master + tserver. Set the replication factor to 6. We need that if the connection between the data centers is lost, both clusters continue to work. Also, if one data center is lost, another cluster could service client requests.
Will this option be fault-tolerant and is it normal to use this approach?

2

Hi @Uladzislau

Can you be more clear why you mean that this is not suitable?

RF must be an odd number, see Deployment checklist for YugabyteDB clusters | YugabyteDB Docs.

This option will require you to have 2 separate clusters and be connected with xCluster xCluster replication (2+ regions) in YugabyteDB | YugabyteDB Docs.

It won’t work with a single cluster with synchronous replication (even if you had 3 regions).

3

It is not convenient that you need to manually or using a script get the IDs of all tables and list them in the setup_universe_replication command. Also, when changing the database schema, you will have to constantly add new tables to replication.

I would like to be able to replicate all databases with all schemas and tables without having to list all the IDs of all tables.

Maybe I missed something?

4

This is in the roadmap, to replicate all tables. DDL conflicts is a separate issue though.

You’ll need 3 DCS with normal synchronous replication.

5

But we only have 2 data centers. A third is not planned.

6

If I get you right. There is no way to replicate 2 data centers with asynchronous replication and not list the IDs of all tables.

7

With the constraints you mentioned (2 dcs being able to function if either one of them goes down) you can only use xCluster.

Not yet.

1 Like
8

Thanks for the info.

9

We will support the database level APIs from the next release
[xCluster] Support Database/Keyspace level replication · Issue #10984 · yugabyte/yugabyte-db · GitHub .

But this is going to be for YSQL only. And it will be transactional uni-direction replication. We do not support Bi-Directional Transactional xCluster since with async replication there is always a lag, and we cannot guarantee consistency when writing to both sides at the same time. You can have 2 DBs, with unidirectional transactional xCluster on different directions and write to the appropriate side.
You will have to run the YSQL DDL change on both sides, but you dont have to worry about getting the ID and adding it to replication. The automatic replication of DDLs itself is planned in [DocDB] xcluster: Automatically propagate DDL changes across clusters · Issue #11537 · yugabyte/yugabyte-db · GitHub

1 Like