Prometheus metrics

General
1

we have new setup and we provided our custom ports as below. When we try to get the prometheus metrics, it shows data only from 1 port (master_webserver_port).

{
“data_dir”: “/data/yugabyte”,
“log_dir”: “/log/yugabyte”,
“ysql_port”: 8090,
“ycql_port”: 8097,
“master_rpc_port”: 8092,
“tserver_rpc_port”: 8093,
“master_webserver_port”: 8094,
“tserver_webserver_port”: 8095,
“ysql_metric_port”: 8096,
“yugabyted_ui_port”:8091,
“enable_pg_parity_early_access”: true,
“certs_dir”: “/data/certs/cluster-name”
}

we are able to view few prometheus metrics using only “master_webserver_port” (8094). But we don’t see all metrics such as “Throughput and latency metrics”

2

Hi @explore_yb

Have you read YugabyteDB metrics | YugabyteDB Docs ?

yb-master is only for metadata, you need to connect to yb-tserver for metrics such as “Throughput and latency metrics" (see metrics endpoints).

3

Hi @dorian_yugabyte
Thank you for the update. do we need to query the standard(default) port for Prometheus ? or these ports are configurable ?
I tried “ysql_metric_port” (8096), but it does not show any data

4

By defaut, you have to scrape these:

image
image860×582 85.3 KB

Looking at the code I think this isn’t being used correctly. Let me ask about this and get back to you.

5

What about <tserver-ip>:8096/prometheus-metrics ?
Or <tserver-ip>:8096/statements?

6

Can you show what it returns exactly? Like do a wget/curl on it and show full response.

7

I tried both url’s, but no response. The connection just timeout after sometime. But I try same curl command with 8094 port, then I get the response

8

@explore_yb Let’s try to figure out if tserver webservice is listening on port `8096’. We can try to run nmap on 8096 and see if you’re able to reach the endpoint.

nmap 8096 <tserver-ip>

What is the version of YugabyteDB you’re using?

Thanks,
Nikhil

9

below is the output of nmap. 8086 port is not working, although all other ports are working.
do you think changing 8086 to any port for “ysql_metric_port” makes sense here ?
Also, do we need to query 8086 for prometheus metrics ?

10

nmap 8096 host-1-ip
Starting Nmap 7.92 ( https://nmap.org ) at 2025-05-09 23:02 UTC
setup_target: failed to determine route to 8096 (0.0.35.136)
Note: Host seems down. If it is really up, but blocking our ping probes, try -Pn
Nmap done: 1 IP address (0 hosts up) scanned in 3.08 seconds

11

Hi Team,
We are also unable to view metrics when using “:13000/metrics” . The nmap get timeout. Is there any custom parameter/config which we need to enable to get YSQL metrics ?
Any pointer would be helpful.

12

What happens when you ssh to the node and run it locally?

13

@explore_yb Are we sure the cluster is healthy?

Can you run the following command to check the status of the cluster?

./bin/yugabyted status --base_dir <path_to_base_dir>

Also, you can navigate to the yugabyted-UI page or master UI (8094) to check the status of the cluster.

Thanks,
Nikhil

14

@nmalladi please find below the cluster status
±-----------------------------------------------------------------------------------------------------------------------------------------------+
| yugabyted |
±-----------------------------------------------------------------------------------------------------------------------------------------------+
| Status : Running. |
| YSQL Status : Ready |
| Replication Factor : 5 |
| Security Features : Encryption-in-transit, Password Authentication |
| Web console : http://:8094 |
| JDBC : jdbc:postgresql://:8090/yugabyte?user=yugabyte&password=<DB_PASSWORD> |
| YSQL : bin/ysqlsh -h -p 8090 -U yugabyte -d yugabyte |
| YCQL : bin/ycqlsh 8097 -u cassandra --ssl |
| Data Dir : /data/yugabyte |
| Log Dir : /log/yugabyte |
| Universe UUID : 07cxx8-2e94-4xx9-bxx4-9exxxx103ac |
±-----------------------------------------------------------------------------------------------------------------------------------------------+

@dorian_yugabyte
I’m able to get ysql metrics with below
curl http://:13000/metrics

but with 8096 port, I get “connection refused”
curl http://:8096/metrics
curl: (7) Failed to connect to port 8096: Connection refused

15

@explore_yb yugabyted can’t modify the ports using that config-name.

You have to set –pgsql-proxy-webserver-port inside --tserver_flags like this:

yugabyted start --tserver_flags="pgsql-proxy-webserver-port=8096"
16

@dorian_yugabyte Thank you for the update. We will try to update the port using the flag and will verify the metrics