Timeout during YCQL Batch Inserts

General
1

Hi there,

I am getting this exception for a batch of 50000 inserts (single node cluster, local laptop):

com.datastax.oss.driver.api.core.DriverTimeoutException: Query timed out after PT2S
at com.datastax.oss.driver.api.core.DriverTimeoutException.copy(DriverTimeoutException.java:34) ~[java-driver-core-4.6.0-yb-11.jar

The timeout appears to be 2 seconds, because if I reduce my batch size to 40000, then I do not get this timeout and the running times I measure for each such batch is always slightly less than 2 seconds.

I have not configured any timeouts myself, so I suppose there is some default value - how can I change that? 2 seconds appears to be awfully little for data ingestion.

My Java code for batch inserting is:

    BatchStatementBuilder builder = new BatchStatementBuilder(BatchType.UNLOGGED);
    PreparedStatement stmt = createPreparedStatementForMyData(...);
    for (Map<String, Object> row : myRows) {
        Object[] values = createValuesForOneRow(...);
        builder.addStatement(stmt.bind(values));
    }
    session.execute(builder.build());

The total running time for data ingestion (if I reduce my batch size enough to avoid the timeout) appears to be pretty much on par with some other databases I tested. But one other thing that I find weird: when I login to the ycqlsh shell, then I will not see any new data being created while logged in - all I ever see is the data that has been present at login.

2

@ulim

Does this help https://support.yugabyte.com/hc/en-us/articles/4403707196557-Getting-error-messages-like-Timed-out-waiting-for-server-response-and-Client-request-timeout- ?

3

This is weird.

How are you verifying this?

And how can we reproduce it?

What do you mean by while being logged in? If you close and re-open ycqlsh it shows up?

YCQL transaction mode is “autocommit”, so there shouldn’t be any state between queries and new data should show up.

4

I saw this article, but thought it didn’t apply to my case. I am seeing a timeout value of 2 seconds, not 10 or 60 as mentioned in this article. Also, in the article it is only explained how to set the timeout for cqlsh, but I need to set it for the Java driver. Also, if that matters, the client (my application) doesn’t read anything, it just INSERTs.

5

About the cqlsh situation: I am starting cqlsh and do “describe tables;”. Then via my application I am creating new tables. My expectation would be that another “describe tables;” would then show the new tables and I could select from them. But in reality I have to restart cqlsh to see the new tables.

6

Can replicate this with 2 ycqlsh clis opened. ycqlsh is probably caching the tables somehow. Can you create an issue on github?

7

cc @nmalladi for answer on java driver

8

Any updates on this? I am getting timeouts left and right, a lot of interesting stuff cannot be done inside 2 seconds.

9

Isn’t 2 seconds the default for write_request_timeout_in_ms ? Like here:

10

Yes, possibly. The question was two-fold:

a) which timeout-related configuration values are available? I believe your link answers that, many thanks for that.
b) how to set these values in the Java driver for Yugabyte?

11

Ok, I got this. There needs to be a file “application.conf” in the classpath and it should contain something like this:

datastax-java-driver {
  profiles {
    MY_PROFILE {
      basic.request.timeout = 25 seconds
    }
  }
}

Then later in the Java code this can be activated by:

SimpleStatement stmt = SimpleStatement.newInstance("select * from ...");
session.execute(stmt.setExecutionProfileName("MY_PROFILE"));