All the transactions hang - not sure whats wrong

General
1

I use spring boot with postgres driver to connect to yugabyte db.

The operation being performed is a set of inserts/updates followed by a transaction rollback. The application supports a concept of dry run mode where all the operations are performed and transaction is rolled back at the end.

In this case; all the connections hang without any progress. Not sure whats wrong.

The tserver logs are as follows

    W1024 12:09:11.682448 10050 transaction_participant.cc:1270] T fd4ef2e5d8b84920bb7266ef1afa12d3 P 0aa55c2f4bd446a9910d06efa155b3a4: Transaction not found: 7864be42-6143-40df-809b-4d022dc31af9, for: conflict resolution
    W1024 12:09:11.682574 10050 transaction_participant.cc:1270] T fd4ef2e5d8b84920bb7266ef1afa12d3 P 0aa55c2f4bd446a9910d06efa155b3a4: Transaction not found: 7864be42-6143-40df-809b-4d022dc31af9, for: get commit time
    W1024 12:09:11.682595 10050 intent_aware_iterator.cc:114] Intent for transaction w/o metadata: 7864be42-6143-40df-809b-4d022dc31af9
    W1024 12:09:11.686857  8666 transaction_participant.cc:1270] T 1452565758364ae8a2b55d7b540c5d17 P 0aa55c2f4bd446a9910d06efa155b3a4: Transaction not found: 7864be42-6143-40df-809b-4d022dc31af9, for: conflict resolution
    W1024 12:09:11.686980  8666 transaction_participant.cc:1270] T 1452565758364ae8a2b55d7b540c5d17 P 0aa55c2f4bd446a9910d06efa155b3a4: Transaction not found: 7864be42-6143-40df-809b-4d022dc31af9, for: get commit time
    W1024 12:09:11.687012  8666 intent_aware_iterator.cc:114] Intent for transaction w/o metadata: 7864be42-6143-40df-809b-4d022dc31af9
    W1024 12:09:11.688719 10053 transaction_participant.cc:1270] T 061cf02ae8d04b7ba0bab3f372dd37eb P 0aa55c2f4bd446a9910d06efa155b3a4: Transaction not found: 7864be42-6143-40df-809b-4d022dc31af9, for: get commit time
    W1024 12:09:11.688781 10053 intent_aware_iterator.cc:114] Intent for transaction w/o metadata: 7864be42-6143-40df-809b-4d022dc31af9


    I1024 12:21:09.534194 19753 log.cc:560] T 3696b16f22c647f2b65917ceb515af47 P 0aa55c2f4bd446a9910d06efa155b3a4: Max segment size 1048576 reached. Starting new segment allocation.
    I1024 12:21:09.567441 19753 log.cc:468] T 3696b16f22c647f2b65917ceb515af47 P 0aa55c2f4bd446a9910d06efa155b3a4: Rolled over to a new segment: /home/srikanth/yugabyte-data/node-1/disk-1/yb-data/tserver/wals/table-00004000000030008000000
    00000409d/tablet-3696b16f22c647f2b65917ceb515af47/wal-000000009
    I1024 12:21:09.572623 19742 log.cc:560] T 2f463539c9174157ab56b661215ec252 P 0aa55c2f4bd446a9910d06efa155b3a4: Max segment size 1048576 reached. Starting new segment allocation.

also ulimit -c is zero.

the java stack is as follows…

 java.lang.Thread.State: RUNNABLE
        at java.net.SocketInputStream.socketRead0(java.base@11.0.2/Native Method)
        at java.net.SocketInputStream.socketRead(java.base@11.0.2/SocketInputStream.java:115)
        at java.net.SocketInputStream.read(java.base@11.0.2/SocketInputStream.java:168)
        at java.net.SocketInputStream.read(java.base@11.0.2/SocketInputStream.java:140)
        at org.postgresql.core.VisibleBufferedInputStream.readMore(VisibleBufferedInputStream.java:140)
        at org.postgresql.core.VisibleBufferedInputStream.ensureBytes(VisibleBufferedInputStream.java:109)
        at org.postgresql.core.VisibleBufferedInputStream.read(VisibleBufferedInputStream.java:67)
        at org.postgresql.core.PGStream.receiveChar(PGStream.java:306)
        at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1952)
        at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:308)
        - locked <0x0000000088100728> (a org.postgresql.core.v3.QueryExecutorImpl)
        at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:441)
        at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:365)
        at org.postgresql.jdbc.PgPreparedStatement.executeWithFlags(PgPreparedStatement.java:143)
        at org.postgresql.jdbc.PgPreparedStatement.executeQuery(PgPreparedStatement.java:106)
        at com.zaxxer.hikari.pool.ProxyPreparedStatement.executeQuery(ProxyPreparedStatement.java:52)
        at com.zaxxer.hikari.pool.HikariProxyPreparedStatement.executeQuery(HikariProxyPreparedStatement.java)

Please let me know the issue

2

Hi @sriki77-

  1. Is there any additional info you can provide about the test (like table schema and the set of DML operations to reproduce the problem)?

  2. What version of YB are you running?

regards,
Kannan

3

I am using the version 2.0.1.0.

Here is one of the tables…others are similar

https://pastebin.com/raw/J9WWUNaR

basically - lots of numeric columns; with serial columns and indexes.

4

Please paste the create table command.
And the queries that you’re running so I can try to reproduce it.

5

After following with the user on another channel, the issue was using HDD disks.
Currently HDD disks aren’t officially supported Deploy | YugabyteDB Docs