Join two cluster and internally on non-standard port numbers

General
1

Hi

I am trying to use a different port to 7100 for each node to get 2 clusters of 3 nodes (each node is an Intel NUC with 64GB RAM and 2TB of disk space) talking to each other over a dynamic dns connection (3 nodes on each side) (basically port forwarding across the connection and thus needing a different port number to identify which node is being communicated with).

I am able to get each side of 3 nodes working via --join parameters being supplied to “yugabyted start” when I used 7000/7100 etc, but as soon as I try to change say the 7100 port number to be unique for each node then I can’t seem to get anywhere.

Is this a scenario that should work?

Thanks

Andrew

2

Hi @andymc

Can you paste some commands that you’re running and errors that you’re getting?
Also, is this for a production setup?

Are you doing xCluster replication between 2 clusters: xCluster | YugabyteDB Docs ?

3

Hi @andymc you are right there is an ugly hardcoded 7100 in https://github.com/yugabyte/yugabyte-db/blob/2.13.2/bin/yugabyted#L2100
I’ll open a git issue

4
5

To recap,

nuc1:
yugabyted start --listen=192.168.0.10

nuc2:
yugabyted start --listen=192.168.0.20 --join=192.168.0.10

nuc3:
yugabyted start --listen=192.168.0.30 --join=192.168.0.10

Can get all those 3 working, tables loading and syncing with each other on one subnet.

Second subnet:

nuc4
yugabyted start --listen=192.168.1.10

nuc5
yugabyted start --listen=192.168.1.20 --join=192.168.1.10

nuc6
yugabyted start --listen=192.168.1.30 --join=192.168.1.10

Then the interface for each side is a dynamic dns inside a telco data centre and to communicate between the two sides I need to use port forwarding and assign each of the 6 Nucs a different port number/numbers so things are routed appropriately.

Thanks Franck - I see your git issue comment. Will follow along and test it again when that gets resolved.

I am very tempted to write some kind of beginner article as it has taken me quite a while to get this far. It was an attempt to move away from SQL Server to something more open source.

Yes, using the preview version.

In response to Dorian, this is a trial of Yugabyte across a couple of different projects/products and probably the final config for at least one of those.

Regards
Andrew

6

I can hard code the relevant replacment value for 7100 in yugabyted for each node for now.

Thanks for the pointing out where it’s happening.

7

@andymc as yugaybted is python, you can change the port number. You can also run in a docker/podman container so that you can remap the ports. I don’t really understand why this port (7100) needs to be tunneled. It is internal to each cluster.

Yes, please. ping me here, e-mail (fpachot at yugabyte.com) or slack forum (Slack) if you need any help, review or promotion

8

Please note that yugabyted cli is not recommended for production deployments: yugabyted reference | YugabyteDB Docs

9

Thanks Dorian for the beta reminder. Will look to using yb-master and yb-server once something is ready for prod.