How to create a multi-cluster deployment on Kubernetes?

General
6

Here is the instructions on how you can run YugaByte db on a regional cluster (multi zone) on GKE. This same instructions can be used for multi region clusters as well.

Pre-Req:

  1. Create a Regional cluster with minimum 4 nodes, with 8 CPU each
  2. Clone YugaByte charts repository: GitHub - yugabyte/charts: Respository of all helm charts that are owned by YugabyteDB for database deployment acroos various K8s configurations
  3. Make sure helm chart 2.8.0+ is installed.

Installation Steps:
Step1: Create Storage Class and Overrides template files
$ mkdir yb-multi-az
$ echo 'kind: StorageClass apiVersion: storage.k8s.io/v1 metadata: name: standard-##zone## provisioner: kubernetes.io/gce-pd parameters: type: pd-standard replication-type: none zone: ##zone##"' > yb-multi-az/storage-class-template.yml

$ echo 'isMultiAz: True AZ: "##zone##" storage: master: storageClass: "standard-##zone##" tserver: storageClass: "standard-##zone##" masterAddresses: "yb-master-0.yb-masters.yb-demo-##zone1##.svc.cluster.local:7100, yb-master-0.yb-masters.yb-demo-##zone2##.svc.cluster.local:7100, yb-master-0.yb-masters.yb-demo-##zone3##.s vc.cluster.local:7100" replicas: master: 1 tserver: 1 totalMasters: 3 gflags: master: placement_cloud: "kubernetes" placement_region: "##region##" placement_zone: "##zone##" tserver: placement_cloud: "kubernetes" placement_region: "##region##" placement_zone: "##zone##"' > yb-multi-az/overrides-template.yml

Step 2: Setup RBAC
$ cd charts/stable/yugabyte/
$ kubectl apply -f yugabyte-rbac.yaml

Step 3: Generate Zone specific overrides
First fetch the zone and region labels by running this command
$ kubectl get nodes -Lfailure-domain.beta.kubernetes.io/region -Lfailure-domain.beta.kubernetes.io/zone

ex: lets say your region label is us-west1 and zone labels are us-west1-a, us-west1-b, us-west1-c
you would generate the zone specific overrides like below

Below command shows for one zone(us-west1-a), Repeat this step for each of the zones.
$ sed 's/##zone##/us-west1-a/g' storage-class-template.yml > yb-multi-az/storage_class_us-west1-a.yml
$ sed 's/##zone##/us-west1-a/g; s/##region##/us-west1/g; s/##zone1##/us-west1-a/g; s/##zone2##/us-west1-b/g; s/##zone3##/us-west1-c/g' overrides-template.yml > yb-multi-az/overrides-us-west1-a.yml

Step 4: Create the storage class
Note: repeat this command for all the zones
$ kubectl applly -f yb-multi-az/storage_class_us-west1-a.yml

Step 5: Initialize helm and have it use the service account we created on Step 2:
$ helm init --service-account yugabyte-helm --upgrade —wait

Step 6: Run helm install.
Note: repeat this command for all the zones
$ helm install charts/stable/yugabyte --namespace yb-demo-us-west1-a --name yb-demo-us-west1-a -f yb-multi-az/overrides-us-west1-a.yml

Step 7: Wait for all the containers to come up
Below command would list all the pods that are specific to yugabytedb
$ kubectl get pods --all-namespaces -lcomponent=yugabytedb

Step 8: Access the db admin UI
Below command would fetch all the services which are expose via loadBalancer.
$ kubectl get svc --all-namespaces -lcomponent=yugabytedb --field-selector=metadata.name=yb-master-ui

If you run into issues please hop on our community slack channel and one of us can help debug