# TransportException: Connection has been closed

**URL:** <https://forum.yugabyte.com/t/transportexception-connection-has-been-closed/347>\
**Category:** General\
**Created:** [May 10, 2019, 12:55pm UTC](https://forum.yugabyte.com/t/transportexception-connection-has-been-closed/347 "2019-05-10T12:55:33Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![apundir](https://avatars.discourse-cdn.com/v4/letter/a/e95f7d/32.png) [@apundir](https://forum.yugabyte.com/u/apundir)\
**Post date:** [May 10, 2019, 12:55pm UTC](https://forum.yugabyte.com/t/transportexception-connection-has-been-closed/347/1 "2019-05-10T12:55:33Z")

</div>

Hi,  
I am doing a POT for YB. I created a 3 node cluster (RF = 3) on local machine (MacBook Pro) as explained in the documentation of YB.  
I have written a java utility to create a key-space with 6 tables. It is a simple utility that does - Cluster → Session → Execute CQL.

The utility fails with the exception trace pasted below. I checked the logs and I see “Too many open files (error 24)”. Not sure if this is the root cause. Just to create a key-space with 6 tables, what’s the kind of settings are we looking on a local machine?

**Local Machine Open Files Configuration:**  
_kern.maxfiles: 49152_  
_kern.maxfilesperproc: 24576_

**Exception Stack Trace:**  
_com.datastax.driver.core.exceptions.TransportException: [/127.0.0.3:9042] Connection has been closed_

- at com.datastax.driver.core.Connection$ConnectionCloseFuture.force(Connection.java:1241)\*
- at com.datastax.driver.core.Connection$ConnectionCloseFuture.force(Connection.java:1226)\*
- at com.datastax.driver.core.Connection.defunct(Connection.java:464)\*
- at com.datastax.driver.core.Connection$ChannelCloseListener.operationComplete(Connection.java:1183)\*
- at com.datastax.driver.core.Connection$ChannelCloseListener.operationComplete(Connection.java:1173)\*
- at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:507)\*
- at io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:500)\*
- at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:479)\*
- at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:420)\*
- at io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:104)\*
- at io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:82)\*
- at io.netty.channel.AbstractChannel$CloseFuture.setClosed(AbstractChannel.java:1004)\*
- at io.netty.channel.AbstractChannel$AbstractUnsafe.doClose0(AbstractChannel.java:633)\*
- at io.netty.channel.AbstractChannel$AbstractUnsafe.close(AbstractChannel.java:611)\*
- at io.netty.channel.AbstractChannel$AbstractUnsafe.close(AbstractChannel.java:554)\*
- at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.closeOnRead(AbstractNioByteChannel.java:71)\*
- at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:158)\*
- at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:643)\*
- at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:566)\*
- at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:480)\*
- at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:442)\*
- at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:131)\*
- at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:144)\*
- at java.lang.Thread.run(Thread.java:748)\*

**Log Excerpt**

_E0510 17:54:20.340422 141635584 tablet.cc:508] T 2bbb83c56968467f8ab5b223b9dcdc2f P 0a6055a954b445e198ea2b225f8a7f8c: Failed to open a RocksDB database in directory /private/tmp/yugabyte-local-cluster/node-2/disk-2/yb-data/tserver/data/rocksdb/table-d626fab88fce4545a488b7687e21d7a4/tablet-2bbb83c56968467f8ab5b223b9dcdc2f: IO error (yb/rocksdb/util/env\_posix.cc:337): /private/tmp/yugabyte-local-cluster/node-2/disk-2/yb-data/tserver/data/rocksdb/table-d626fab88fce4545a488b7687e21d7a4/tablet-2bbb83c56968467f8ab5b223b9dcdc2f: Too many open files_

_E0510 17:54:20.341192 143245312 ts\_tablet\_manager.cc:963] T c9bb00c885bf4bcb998d2453687efcf2 P 0a6055a954b445e198ea2b225f8a7f8c: Tablet failed to bootstrap: Illegal state (yb/tablet/tablet.cc:513): IO error (yb/rocksdb/util/env\_posix.cc:225): /private/tmp/yugabyte-local-cluster/node-2/disk-1/yb-data/tserver/data/rocksdb/table-d626fab88fce4545a488b7687e21d7a4/tablet-c9bb00c885bf4bcb998d2453687efcf2/MANIFEST-000001: Too many open files_

_E0510 17:54:20.341200 141635584 ts\_tablet\_manager.cc:963] T 2bbb83c56968467f8ab5b223b9dcdc2f P 0a6055a954b445e198ea2b225f8a7f8c: Tablet failed to bootstrap: Illegal state (yb/tablet/tablet.cc:513): IO error (yb/rocksdb/util/env\_posix.cc:337): /private/tmp/yugabyte-local-cluster/node-2/disk-2/yb-data/tserver/data/rocksdb/table-d626fab88fce4545a488b7687e21d7a4/tablet-2bbb83c56968467f8ab5b223b9dcdc2f: Too many open files_

_W0510 17:54:20.343843 144318464 db\_impl.cc:5823] T 6e70f835dc6c4580aea19b8177f39113: Unable to persist options in DB::Open() – IO error (yb/rocksdb/util/env\_posix.cc:225): /private/tmp/yugabyte-local-cluster/node-2/disk-1/yb-data/tserver/data/rocksdb/table-d626fab88fce4545a488b7687e21d7a4/tablet-6e70f835dc6c4580aea19b8177f39113/OPTIONS-000004.dbtmp: Too many open files_

_W0510 17:54:20.345109 144318464 log.cc:1086] Error closing log: Illegal state (yb/consensus/log.cc:938): Bad state for Close() 0_

_E0510 17:54:20.345659 144318464 ts\_tablet\_manager.cc:963] T 6e70f835dc6c4580aea19b8177f39113 P 0a6055a954b445e198ea2b225f8a7f8c: Tablet failed to bootstrap: IO error (yb/util/env\_posix.cc:1093): Failed to open new log: Unable to read children from path: /private/tmp/yugabyte-local-cluster/node-2/disk-1/yb-data/tserver/wals/table-d626fab88fce4545a488b7687e21d7a4/tablet-6e70f835dc6c4580aea19b8177f39113: Too many open files (error 24)_

_E0510 17:54:20.347438 143781888 ts\_tablet\_manager.cc:985] T d2a6a67f0cc44d4bbaf86fa1f0e36ac4 P 0a6055a954b445e198ea2b225f8a7f8c: Tablet failed to init: IO error (yb/util/env\_posix.cc:1013): /private/tmp/yugabyte-local-cluster/node-2/disk-1/yb-data/tserver/consensus-meta/d2a6a67f0cc44d4bbaf86fa1f0e36ac4: Too many open files (error 24)_

_E0510 17:54:20.347455 142708736 ts\_tablet\_manager.cc:985] T 400658370bbb4ae78f2a9e4b7d7165d1 P 0a6055a954b445e198ea2b225f8a7f8c: Tablet failed to init: IO error (yb/util/env\_posix.cc:1013): /private/tmp/yugabyte-local-cluster/node-2/disk-1/yb-data/tserver/consensus-meta/400658370bbb4ae78f2a9e4b7d7165d1: Too many open files (error 24)_

---

<div class="post-metadata">

**Author:** ![Alan\_Caldera](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.yugabyte.com/alan_caldera/32/91_2.png) [@Alan\_Caldera](https://forum.yugabyte.com/u/Alan_Caldera)\
**Post date:** [May 10, 2019, 1:20pm UTC](https://forum.yugabyte.com/t/transportexception-connection-has-been-closed/347/2 "2019-05-10T13:20:04Z")

</div>

You should increase your kern.maxfiles to 1048576. Here is what my ulimits are:

```auto
Alans-MacBook-Pro-2:~ acaldera$ ulimit -a
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
file size (blocks, -f) unlimited
max locked memory (kbytes, -l) unlimited
max memory size (kbytes, -m) unlimited
open files (-n) 1048576
pipe size (512 bytes, -p) 1
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) 1418
virtual memory (kbytes, -v) unlimited

```

---

<div class="post-metadata">

**Author:** ![apundir](https://avatars.discourse-cdn.com/v4/letter/a/e95f7d/32.png) [@apundir](https://forum.yugabyte.com/u/apundir)\
**Post date:** [May 13, 2019, 6:43pm UTC](https://forum.yugabyte.com/t/transportexception-connection-has-been-closed/347/3 "2019-05-13T18:43:11Z")

</div>

> [@Alan\_Caldera](#):
>
> 1048576

Thanks Alan, it worked.
