Cannot connect ysql

General
1

Have a 4 node deployment, have configured master on 3 nodes and tablet servers on all 4, YCQL is working but cannot use ysqlsh am getting the following error:
ysqlsh: server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.

tserver log files have nothing about the error, postgres is running, as follows:

/opt/yugabyte-1.3.0.0/ post gres/bin/ post gres -D /mnt/yugga_vol01/pg_data -p 5433 -h 0.0.0.0 -k -c logging_collector=on -c log_directory=/mnt/yugga_vol01/yb-data/tserver/logs -c shared_preload_libraries=yb_pg_metrics -c yb_pg_metrics.node_name=DEFAULT_NODE_NAME -c yb_pg_metrics.port=13000

How do I debug this further?

2

Hi @salil,
Did you have an older version of YB and then upgrade/install a newer version? We’ve had a backward incompatible change in our recent version for YSQL and the error you’re seeing above is a typical symptom when you upgrade.

If this is a fresh install (and not an upgrade), then you can try this to debug:

  1. Update postgresql.conf file located in $data_dir/disk-1/pg_data/postgresql.conf and add the line log_statement='all' . Please do this on the node you’re connecting to with ysqlsh .
  2. Restart tserver on that node
  3. Run ./bin/ysqlsh and check any errors in $data_dir/disk-1/yb-data/tserver/logs/postgresql*.log
3

Thanks @neha. I was using the flagfile option for yb-tserver, but was not giving the full path of the file. So postgres was complaining about tsserver.conf not found. I restarted tserver with full path and now I am getting a different error:

ysqlsh: could not connect to server: Connection refused
Is the server running on host “localhost” (::1) and accepting
TCP/IP connections on port 5433?
FATAL: Not found: Error loading table with oid 1260 in database with oid 1: The object does not exist: table_id: “000000010000300080000000000004ec”

4

Hi @salil, I think this is because you haven’t yet initialized YSQL on the cluster. Please follow the instructions here to do that:

You should be able to connect after that.

5

Hi @neha,
Thanks, this did help me get it working (at least can get to the ysql shell now). However the instructions as documented did not help. .

Currently the steps given are:
-start tserver
-init ysql.

However I noticed the following:

  • when I start tserver, it starts postgres with a data dir in $data_dir/disk-1/pg_data
  • The instructions ask you to init the ysql using the following data dir:
    /tmp/yb_pg_initdb_tmp_data_dir

Doing this does not work as tserver initiated postgress process is not looking at this data dir!

I tried to run the initdb again giving $data_dir/disk-1/pg_data as data dir, however this command failed saying the dir already exists.

So here are the steps I performed to get ysql working:

  • killed tserver
  • removed $data_dir/disk-1/pg_data dir
  • ran initdb script with -D $data_dir/disk-1/pg_data
  • While the script was running, started tserver in another window
6

Glad to hear that it’s working for you now, @salil. And thank you for your feedback on the documentation. We’ll be sure to review and update it.

7

Hi Neha,
I am using v yugabyte-1.2.6.0. Installed 3 VMs ( with 3 Master and 3 tserver ) running.
I am using the below CLI for ybmaster and ybtserver

./bin/yb-master --master_addresses 192.168.56.11:7100,192.168.56.12:7100,192.168.56.13:7100 --rpc_bind_addresses 192.168.56.13 --fs_data_dirs=/home/yugabyte/datadrive >& /home/yugabyte/datadrive/yb-master.out &

./bin/yb-tserver --tserver_master_addrs 192.168.56.11:7100,192.168.56.12:7100,192.168.56.13:7100 --rpc_bind_addresses 192.168.56.11 --start_pgsql_proxy --pgsql_proxy_bind_address=192.168.56.11:5433 --cql_proxy_bind_address=192.168.56.11:9042 --fs_data_dirs “/home/yugabyte/datadrive” >& /home/yugabyte/datadrive/yb-tserver.out &

Cluster setup is fine but unable to connect the sever through client. Getting below errors when connecting from psql client.

psql: error: connection to server at “192.168.56.12”, port 5433 failed: FATAL: Not found: Error loading table with oid 1260 in database with oid 1: The object does not exist: table_id: “000000010000300080000000000004ec”

Corrosponing postgreSQl log file error

2022-01-05 13:24:48.364 UTC [3363] FATAL: Not found: Error loading table with oid 1260 in database with oid 1: The object does not exist: table_id: “000000010000300080000000000004ec”

Also I am not able to find ysqlsh.
It is not there in my tar ball. I am using the latest one yugabyte-1.2.6.0.
root@sachin-VirtualBox1:~/yugabyte-1.2.6.0# ls -ltr bin
total 32288
lrwxrwxrwx 1 sachin yugabyte 20 Apr 21 2019 psql → …/postgres/bin/psql
-rwxrwxr-x 1 sachin yugabyte 3692392 Jan 5 09:30 yb_load_test_tool
-rwxrwxr-x 1 sachin yugabyte 3376192 Jan 5 09:30 yb-tserver
-rwxrwxr-x 1 sachin yugabyte 1829936 Jan 5 09:30 log-dump
-rwxrwxr-x 1 sachin yugabyte 2744384 Jan 5 09:30 yb-master
-rwxr-xr-x 1 sachin yugabyte 3808 Jan 5 09:30 post_install.sh
-rwxr-xr-x 1 sachin yugabyte 9093 Jan 5 09:30 yb-prof.py
-rwxr-xr-x 1 sachin yugabyte 2327 Jan 5 09:30 bulk_load_cleanup.sh
-rwxr-xr-x 1 sachin yugabyte 2468464 Jan 5 09:30 redis-cli
-rwxr-xr-x 1 sachin yugabyte 1516 Jan 5 09:30 configure
-rwxrwxr-x 1 sachin yugabyte 5594088 Jan 5 09:30 yb-admin
-rwxr-xr-x 1 sachin yugabyte 178257 Jan 5 09:30 pprof
-rwxr-xr-x 1 sachin yugabyte 4056 Jan 5 09:30 bulk_load_helper.sh
-rwxr-xr-x 1 sachin yugabyte 1141216 Jan 5 09:30 patchelf
-rwxrwxr-x 1 sachin yugabyte 2905848 Jan 5 09:30 yb-ysck
-rwxr-xr-x 1 sachin yugabyte 97735 Jan 5 09:30 cqlsh.py
-rwxrwxr-x 1 sachin yugabyte 260920 Jan 5 09:30 yb-pbc-dump
-rwxrwxr-x 1 sachin yugabyte 21096 Jan 5 09:30 sst_dump
-rwxr-xr-x 1 sachin yugabyte 1353 Jan 5 09:30 cqlsh
-rwxrwxr-x 1 sachin yugabyte 785704 Jan 5 09:30 yb-generate_partitions_main
-rwxrwxr-x 1 sachin yugabyte 3307352 Jan 5 09:30 yb-ts-cli
-rwxr-xr-x 1 sachin yugabyte 52027 Jan 5 09:30 yb-ctl
-rwxrwxr-x 1 sachin yugabyte 4544840 Jan 5 09:30 yb-bulk_load

Do I need to initialize ysql seperately. I could not find the same on this page
Start YB-TServer servers | YugabyteDB Docs

Please help !!!

8

Version 1.2.6 is very, very out of date. Are you able to run a current version, instead? The download site has instructions.

If you’re not able to get this working, please start a new thread and we can help you out!

9

Thanks Alex,
I realized that I have downloaded the wrong version . Not sure from where I came across this version. Maybe I picked it from some blog accidently.
I have now installed the latest version and is all good now. Thanks !!