Enabling Encryption in Transit and Password Authentication...Could not update Postgress user password. Exception: Traceback (most recent call last)

General
1

I want to create a multi-node cluster with RF=3. First I created node server certificates:

./bin/yugabyted cert generate_server_certs --hostnames=192.168.198.35,192.168.198.36,192.168.198.37

cp $HOME/var/generated_certs/192.168.198.35/* $HOME/yugabyte-2.21.1.0/node1/certs

./bin/yugabyted start --secure --advertise_address=192.168.198.35
–base_dir=/root/node1
–cloud_location=aws.us-east-1.us-east-1a

ERROR:

YugabyteDB Started
\ Enabling Encryption in Transit and Password Authentication…Could not update Cassandra user password. Exception: Traceback (most recent call last):
File “/root/yugabyte-2.21.1.0/./bin/yugabyted”, line 4478, in update_db_passwords
retry_op_with_argument(ycql_proxy.try_update_password, new_password, timeout=60)
File “/root/yugabyte-2.21.1.0/./bin/yugabyted”, line 8501, in retry_op_with_argument
raise RuntimeError(“Failed after retrying operation for {} secs.”.format(
RuntimeError: Failed after retrying operation for 60.53712773323059 secs.

Please help! How to solve this error? OS is ubuntu 24.04
Thanks!

2

Hi @Farrux

Please run the collect_logs command and upload the logs.

3

Hi @dorian_yugabyte !

./bin/yugabyted collect_logs
ERROR: No YugabyteDB node is running in the data_dir /root/var/data
For more information, check the logs in /root/var/logs

tail /root/var/logs/yugabyted.log
[yugabyted generate_server_certs] 2024-07-01 13:11:31,230 INFO: | 0.0s | Found directory /root/yugabyte-2.21.1.0/bin for file openssl_proxy.sh
[yugabyted generate_server_certs] 2024-07-01 13:11:31,230 INFO: | 0.0s | Found directory /root/yugabyte-2.21.1.0/bin for file yb-admin
[yugabyted generate_server_certs] 2024-07-01 13:11:31,231 INFO: | 0.0s | Found an existing root certs directory /root/var/generated_certs/root-ca. Found appropriate files. Using the existing root certs.
[yugabyted generate_server_certs] 2024-07-01 13:11:31,232 INFO: | 0.0s | :warning: WARNING: Hostnames 192.168.198.35,192.168.198.36,192.168.198.37 already have their certifcates generated according to root-ca certs database index. Please use the previously generated certs. Skipping these hostnames…
[yugabyted collect_logs] 2024-07-01 13:14:44,178 INFO: | 0.0s | Running yugabyted command: ‘./bin/yugabyted collect_logs’
[yugabyted collect_logs] 2024-07-01 13:14:44,178 INFO: | 0.0s | cmd = collect_logs using config file: /root/var/conf/yugabyted.conf
[yugabyted collect_logs] 2024-07-01 13:14:44,178 INFO: | 0.0s | Found directory /root/yugabyte-2.21.1.0/bin for file openssl_proxy.sh
[yugabyted collect_logs] 2024-07-01 13:14:44,179 INFO: | 0.0s | Found directory /root/yugabyte-2.21.1.0/bin for file yb-admin
[yugabyted collect_logs] 2024-07-01 13:14:44,179 ERROR: | 0.0s | ERROR: No YugabyteDB node is running in the data_dir /root/var/data
For more information, check the logs in /root/var/logs

4

Use the same --base_dir that you used to start the cluster.

5

Thanks @dorian_yugabyte

[quote=“dorian_yugabyte, post:4, topic:2756”]
Use the same --base_dir that you used to start the cluster.
[/quote]

In insecure mode it is working, but in secure mode with the key “–secure” giving this error:

tail /root/var/data/logs/yugabyted.log
<<
YugabyteDB Started
| Enabling Encryption in Transit and Password Authentication…Could not update Postgress user password. Exception: Traceback (most recent call last):
File “/root/yugabyte-2.21.1.0/./bin/yugabyted”, line 4468, in update_db_passwords
retry_op_with_argument(ysql_proxy.try_update_password, new_password, timeout=60)
File “/root/yugabyte-2.21.1.0/./bin/yugabyted”, line 8501, in retry_op_with_argument
raise RuntimeError(“Failed after retrying operation for {} secs.”.format(
RuntimeError: Failed after retrying operation for 60.53805756568909 secs.

I ran:

./bin/yugabyted cert generate_server_certs --hostnames=192.168.198.35,192.168.198.36,192.168.198.37

cp $HOME/var/generated_certs/192.168.198.35/* $HOME/var/data/certs

./bin/yugabyted start --secure --advertise_address=192.168.198.35
–base_dir=/root/var/data
–cloud_location=aws.us-east-1.us-east-1a

How to solve this error? Please help!
Thanks!

6

@Farrux please upload the logs like I requested, upload the zip file.

Also, please show the steps you took and guide you followed. Example did you try following this guide yugabyted reference | YugabyteDB Docs ?

7

yes!
Without --secure working, but with it not running.
As a zip uploading not allowed put screenshots:

image
image1900×765 221 KB

image
image1911×781 243 KB

image
image1540×899 157 KB

Thanks!

8

Please upload to a google driver, mediafire, etc.

Did you retry the steps that I sent in the previous reply?

9

logs.zip

yes

10

@Farrux

Please destroy the cluster. Start a new one like it’s said in yugabyted reference | YugabyteDB Docs

And if it fails again, please describe on which step it failed, and upload the logs as generated by yugabyted collect_logs command.

11

Hi @dorian_yugabyte!

image
image1372×897 174 KB

image
image1884×749 170 KB

I ran all commands from yugabyte documenation:
1#./bin/yugabyted cert generate_server_certs --hostnames=192.168.198.35,192.168.198.36,192.168.198.37
2#.cp /root/var/generated_certs/192.168.198.35/* node1/certs
3#…/bin/yugabyted start --secure --advertise_address=192.168.198.35 --base_dir=/root/yugabyte-2.21.1.0/node1 --cloud_location=aws.us-east-1.us-east-1a
But yugabyte cannot run.

12

Please use ./bin/yugabyted collect_logs --base_dir, using base_dir of the installation that failed.

I don’t see --join in your second node, the one that failed in the screenshot above.

13

image
image756×368 82.1 KB

I didn’t ran on 2,3 node because firts one is not up.

14

Let me check internally about this and get back to you.

1 Like
15

Hi @Farrux , I see that different --base_dir ($HOME/var, $HOME/var/data, $HOME/yugabyte-2.21/node1) are being used multiple times. Can you please destroy all the nodes started in all of the base_dir you used so as to create a fresh environment.

Destroy the nodes:
./bin/yugabyted destroy --base_dir=<base_dir>
Please run the above command for all the base dirs you may have used to delete any garbage values remaining

Steps for new node:

  1. Create the certs.
  2. Copy the certs to <base_dir>/certs. In some of the above deployments I see the certs are copied to a different path.
  3. Start the node using --secure and --base_dir flag. --base_dir should be the same as in Step 2.
  4. If any error is thrown. Please run ./bin/yugabyted collect_logs --base_dir=<base_dir> Please the base_dir same. Please upload the zip file created after this step to a cloud location and paste the link here.

Thanks.

16

Hi!

image
image1575×498 98.1 KB

image
image1309×394 86.4 KB

I deleted $HOME/yugabyte-2.21.1.0/ folder and downloaded a new project (Debian 12)

wget https://downloads.yugabyte.com/releases/2.21.1.0/yugabyte-2.21.1.0-b271-linux-x86_64.tar.gz

All my commands:

#./bin/yugabyted stop
#./bin/yugabyted destroy --base_dir=./node1/
#./bin/yugabyted cert generate_server_certs --hostnames=192.168.198.35,192.168.198.36,192.168.198.37 --base_dir=./node1
#cp ./node1/generated_certs/192.168.198.35/* ./node1/certs/
#./bin/yugabyted start --secure --advertise_address=192.168.198.35 --base_dir=$HOME/yugabyte-2.21.1.0/node1 --cloud_location=aws.us-east-1.us-east-1a

And tried with it but it is to gave the error.
Thanks!

17

Hi,

Sorry, we couldn’t respond to you earlier. Please tar.gz the base directory $HOME/yugabyte-2.21.1.0/node1 and send it to us for further review.

Thanks,
Nikhil

18

Hi @nmalladi !

Thanks!

19

I see this error in the logs:

Traceback (most recent call last):
  File "/root/yugabyte-2.21.1.0/bin/ycqlsh.py", line 147, in <module>
    from six.moves import configparser, input
ModuleNotFoundError: No module named 'six.moves'

What is your OS?
Please paste output of:

python3 --version

and

python --version
20

python3 --version
Python 3.11.2

python --version
Python 3.11.2