trying to set up a instance on a AlmaLinux 8 headless system.
I can get yb_master and yb_tserver to run and accept ysqlsh connections from localhost. ss -tuln shows port 5433 listening on 127.0.0.1. if I pass pgsql_proxy_bind_address 0.0.0.0:5433 or just 0.0.0.0 to yb_tserver I get an error message
FATAL: could not open lock file "/tmp/.yb.8228272234018189749/.s.PGSQL.5433.lock": Permission denied
I noticed on document that said it needed the ysql_hba_conf_cv param first which I did and that didn’t help either
at the moment i am passing a flagfile to tserver which is --tserver_master_addrs=127.0.0.1:7100 --rpc_bind_addresses=0.0.0.0:9100 --webserver_interface=0.0.0.0 --ysql_hba_conf_csv="host all yugabyte 127.0.0/1/0 trust, host all all 0.0.0.0/0 md5" --pgsql_proxy_bind_address=0.0.0.0 --enable_ysql --fs_data_dirs=/opt/yugabyte/data/
and I get the error, without the pgsql_proxy_bind_address it starts
Does “/tmp/.yb.8228272234018189749/.s.PGSQL.5433.lock” exist? Does “/tmp/.yb.8228272234018189749/” exist? What are the permissions on them? Is any other process using them: lsof “/tmp/.yb.8228272234018189749/.s.PGSQL.5433.lock”?
The .lock file does not exist, the /tmp/.yb.xxxxxxx folder does exist and is owned by the ‘yugabyte’ user that I am trying to run as with folder permissions of 0755
Even if the permissions seem fine, maybe try removing that directory? I don’t have issues when using your flags.
cat flagfile:
–tserver_master_addrs=127.0.0.1:7100
–rpc_bind_addresses=0.0.0.0:9100
–webserver_interface=0.0.0.0
–ysql_hba_conf_csv=host all yugabyte 127.0.0.1/0 trust,host all all 0.0.0.0/0 md5
–pgsql_proxy_bind_address=0.0.0.0
–enable_ysql=true
–fs_data_dirs=/opt/yugabyte/data/
yb-tserver -flagfile flagfile:
2022-01-20 13:12:13.703 PST [8180] LOG: listening on IPv4 address “0.0.0.0”, port 5433
2022-01-20 13:12:13.707 PST [8180] LOG: listening on Unix socket “/tmp/.yb.8228272234018189749/.s.PGSQL.5433”
I followed the quick start, and I started with yugabyte-2.11.1.0-b305-linux-x86_64.tar.gz
I spun up a AlmaLinux 8.5 x64 minimal install virtual machine to test on
I deleted the .yb.xxxxx in /tmp and it still doesn’t start. following the system calls it seems to go bad on this one ─1446 /opt/yugabyte/postgres/bin/postgres -D /opt/yugabyte/data//pg_data -p 5433 -h 0.0.0.0 -k /tmp/.yb.8228272234018189749 -c unix_socket_permissions=0700 -c logging_collector=on -c log_directory=/opt/yugabyte/data/yb-data/tserver/logs -c yb_pg_metrics.node_name=yugatest.pai.inc:9000 -c yb_pg_metrics.port=13000 -c config_file=/opt/yugabyte/data//pg_data/ysql_pg.conf -c hba_file=/opt/yugabyte/data//pg_data/ysql_hba.conf
I tried with SELinux on or off, no difference
it is just spewing logs, I can’t even scroll to the beginning of an ls in /opt/yugabyte/data/yb-data/tserver/logs
the only thing in yb-tserver.ERROR is Application fingerprint: version 2.11.1.0 build 305 revision 3e7bd6c13de6d0b68e8c4f2b116aba029554e3ee build_type RELEASE built at 14 Dec 2021 05:45:12 UTC Running duration (h:mm:ss): 0:00:10 Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg E0121 12:20:40.163514 1355 call_home.cc:426] Unable to parse json. Error: 6 at offset 32043 in string host all y
yb-tserver.WARNING shows W0121 12:34:14.681875 1358 pg_wrapper.cc:714] PostgreSQL server exited with code 256 W0121 12:34:14.920791 1358 pg_wrapper.cc:714] PostgreSQL server exited with code 256 W0121 12:34:15.161873 1358 pg_wrapper.cc:714] PostgreSQL server exited with code 256 W0121 12:34:15.405970 1358 pg_wrapper.cc:714] PostgreSQL server exited with code 256 W0121 12:34:15.645493 1358 pg_wrapper.cc:714] PostgreSQL server exited with code 256 W0121 12:34:15.882550 1358 pg_wrapper.cc:714] PostgreSQL server exited with code 256 W0121 12:34:16.131435 1358 pg_wrapper.cc:714] PostgreSQL server exited with code 256 W0121 12:34:16.365181 1358 pg_wrapper.cc:714] PostgreSQL server exited with code 256 W0121 12:34:16.606982 1358 pg_wrapper.cc:714] PostgreSQL server exited with code 256 W0121 12:34:16.845465 1358 pg_wrapper.cc:714] PostgreSQL server exited with code 256 W0121 12:34:17.080324 1358 pg_wrapper.cc:714] PostgreSQL server exited with code 256 W0121 12:34:17.316731 1358 pg_wrapper.cc:714] PostgreSQL server exited with code 256 W0121 12:34:17.565565 1358 pg_wrapper.cc:714] PostgreSQL server exited with code 256
and yb-tserver.INFO just repeats W0121 12:35:01.718078 1358 pg_wrapper.cc:714] PostgreSQL server exited with code 256 I0121 12:35:01.718137 1358 pg_wrapper.cc:730] Restarting PostgreSQL server I0121 12:35:01.722963 1358 pg_wrapper.cc:438] PostgreSQL server running as pid 8915 W0121 12:35:01.967739 1358 pg_wrapper.cc:714] PostgreSQL server exited with code 256 I0121 12:35:01.967790 1358 pg_wrapper.cc:730] Restarting PostgreSQL server I0121 12:35:01.972692 1358 pg_wrapper.cc:438] PostgreSQL server running as pid 8917 W0121 12:35:02.211083 1358 pg_wrapper.cc:714] PostgreSQL server exited with code 256 I0121 12:35:02.211143 1358 pg_wrapper.cc:730] Restarting PostgreSQL server I0121 12:35:02.216239 1358 pg_wrapper.cc:438] PostgreSQL server running as pid 8919 W0121 12:35:02.447405 1358 pg_wrapper.cc:714] PostgreSQL server exited with code 256 I0121 12:35:02.447460 1358 pg_wrapper.cc:730] Restarting PostgreSQL server I0121 12:35:02.452747 1358 pg_wrapper.cc:438] PostgreSQL server running as pid 8921
yb-tserver.ERROR Application fingerprint: version 2.8.1.0 build 37 revision 23813c9df3ef689370f4c8a2eb46087d4feb46b6 build_type RELEASE built at 21 Dec 2021 12:00:29 UTC Running duration (h:mm:ss): 0:00:10 Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg E0121 20:28:54.792765 6752 call_home.cc:427] Unable to parse json. Error: 6 at offset 31763 in string host all y
yb-tserver.WARNING W0121 20:33:44.496948 6778 pg_wrapper.cc:706] PostgreSQL server exited with code 256 W0121 20:33:44.861521 6778 pg_wrapper.cc:706] PostgreSQL server exited with code 256 W0121 20:33:45.126411 6778 pg_wrapper.cc:706] PostgreSQL server exited with code 256 W0121 20:33:45.424388 6778 pg_wrapper.cc:706] PostgreSQL server exited with code 256 W0121 20:33:45.655624 6778 pg_wrapper.cc:706] PostgreSQL server exited with code 256 W0121 20:33:45.907779 6778 pg_wrapper.cc:706] PostgreSQL server exited with code 256 W0121 20:33:46.150913 6778 pg_wrapper.cc:706] PostgreSQL server exited with code 256 W0121 20:33:46.380635 6778 pg_wrapper.cc:706] PostgreSQL server exited with code 256 W0121 20:33:46.638939 6778 pg_wrapper.cc:706] PostgreSQL server exited with code 256 W0121 20:33:46.884662 6778 pg_wrapper.cc:706] PostgreSQL server exited with code 256
yb-tserver.INFO I0121 20:34:18.916746 6778 pg_wrapper.cc:722] Restarting PostgreSQL server I0121 20:34:18.921118 6778 pg_wrapper.cc:434] PostgreSQL server running as pid 16566 W0121 20:34:19.168999 6778 pg_wrapper.cc:706] PostgreSQL server exited with code 256 I0121 20:34:19.169062 6778 pg_wrapper.cc:722] Restarting PostgreSQL server I0121 20:34:19.173972 6778 pg_wrapper.cc:434] PostgreSQL server running as pid 16594 W0121 20:34:19.402140 6778 pg_wrapper.cc:706] PostgreSQL server exited with code 256 I0121 20:34:19.402199 6778 pg_wrapper.cc:722] Restarting PostgreSQL server I0121 20:34:19.406625 6778 pg_wrapper.cc:434] PostgreSQL server running as pid 16620 W0121 20:34:19.660766 6778 pg_wrapper.cc:706] PostgreSQL server exited with code 256 I0121 20:34:19.660826 6778 pg_wrapper.cc:722] Restarting PostgreSQL server I0121 20:34:19.665196 6778 pg_wrapper.cc:434] PostgreSQL server running as pid 16644 W0121 20:34:19.892814 6778 pg_wrapper.cc:706] PostgreSQL server exited with code 256 I0121 20:34:19.892874 6778 pg_wrapper.cc:722] Restarting PostgreSQL server I0121 20:34:19.897320 6778 pg_wrapper.cc:434] PostgreSQL server running as pid 16668 W0121 20:34:20.122459 6778 pg_wrapper.cc:706] PostgreSQL server exited with code 256 I0121 20:34:20.122517 6778 pg_wrapper.cc:722] Restarting PostgreSQL server I0121 20:34:20.126960 6778 pg_wrapper.cc:434] PostgreSQL server running as pid 16692 W0121 20:34:20.352560 6778 pg_wrapper.cc:706] PostgreSQL server exited with code 256 I0121 20:34:20.352622 6778 pg_wrapper.cc:722] Restarting PostgreSQL server
one of the postgres logs shows 2022-01-21 20:37:26.381 UTC [21815] FATAL: could not load pg_hba.conf 2022-01-21 20:37:26.382 UTC [21815] LOG: database system is shut down 2022-01-21 20:37:26.605 UTC [21817] LOG: multiple values specified for authentication type 2022-01-21 20:37:26.605 UTC [21817] HINT: Specify exactly one authentication type per line. 2022-01-21 20:37:26.605 UTC [21817] CONTEXT: line 4 of configuration file "/opt/yugabyte/data/pg_data/ysql_hba.conf" 2022-01-21 20:37:26.605 UTC [21817] FATAL: could not load pg_hba.conf 2022-01-21 20:37:26.607 UTC [21817] LOG: database system is shut down 2022-01-21 20:37:26.841 UTC [21819] LOG: multiple values specified for authentication type 2022-01-21 20:37:26.841 UTC [21819] HINT: Specify exactly one authentication type per line. 2022-01-21 20:37:26.841 UTC [21819] CONTEXT: line 4 of configuration file "/opt/yugabyte/data/pg_data/ysql_hba.conf" 2022-01-21 20:37:26.841 UTC [21819] FATAL: could not load pg_hba.conf 2022-01-21 20:37:26.841 UTC [21819] LOG: database system is shut down
and the flag file is --tserver_master_addrs=127.0.0.1:7100 --rpc_bind_addresses=0.0.0.0:9100 --webserver_interface=0.0.0.0 --ysql_hba_conf_csv="host all yugabyte 127.0.0.1/0 trust, host all all 0.0.0.0/0 md5" --pgsql_proxy_bind_address=0.0.0.0:5433 --enable_ysql --fs_data_dirs=/opt/yugabyte/data/
which I almost copied the ysql_hba_conf_csv from the docs
Seems like something went wrong with your hba conf, so this is a different issue from the original socket one. Maybe the flagfile syntax is wrong? I think there’s a difference between pasting those flags directly instead of using a flagfile. The flagfile I used had no double quotes and each flag on a separate line. You can also look at/show the contents of the hba conf file in the pg_data directory.
You are right, and I think I kind of zoned out somewhere because everything was showing the same symptoms. So… this morning I started with a fresh Alma 8 VM , and started from scratch. I cut the parameters down to the bare minimum and everything started. I think I was changing too many things at once to be valid. It is listening but every create command in ysqlsh or a graphic client times out, so that is a different problem.
Thank you again for all the help. I am so much more aware of where all the places to look are when things aren’t working