Repository navigation
FD Leak When Repeatedly Creating Native SSHClient Instances #427
Description
_For general questions please use GitHub Discussions.
Describe the bug
Repeatedly creating and destroying pssh.clients.native.single.SSHClient instances leaks one file descriptor per client lifecycle.
This occurs even when:
keepalive_seconds=0
no commands are executed
no Session, Channel, or KeepAlive objects remain reachable from Python
gc.collect() is forced between iterations
Reusing a single SSHClient instance does not exhibit the leak.
To Reproduce
Steps to reproduce the behavior:
Here's a short script to reproduce the behavior. The USER, HOST, and PKEY values at the beginning of the script will need to be updated:
import gc
import os
import psutil
from pssh.clients.native.single import SSHClient
HOST = "host"
USER = "user"
PKEY = "~/.ssh/id_rsa"
process = psutil.Process(os.getpid())
for i in range(100):
client = SSHClient(
HOST,
user=USER,
pkey=PKEY,
keepalive_seconds=0,
)
del client
gc.collect()
if i % 10 == 0:
print(
f"iter={i} "
f"fds={process.num_fds()}"
)
Expected behavior
After an SSHClient instance becomes unreachable and is garbage collected: all underlying sockets/file descriptors should be closed and FD count should remain constant regardless of the number of client instances created over time.
Acutal behavior
FD count increases roughly one descriptor per iteration:
iter=0 fds=7
iter=10 fds=17
iter=20 fds=27
iter=30 fds=37
iter=40 fds=47
...
iter=100 fds=106
At the end of execution:
import gc
from ssh2.session import Session
from ssh2.channel import Channel
from pssh.clients.native.single import KeepAlive
print(sum(isinstance(o, Session) for o in gc.get_objects()))
print(sum(isinstance(o, Channel) for o in gc.get_objects()))
print(sum(isinstance(o, KeepAlive) for o in gc.get_objects()))
produces:
Session objects=0
Channel objects=0
KeepAlive objects=0
Despite this, the file descriptors remain open.
Inspecting /proc/self/fd shows a large number of leaked socket descriptors.
Activity
pkittenis commentedon Oct 20, 2026
Thanks for the interest and report.
Have a look at
netstat -a | grep <HOST>output after the script exits. It should sayTIME_WAITin the socket status.TIME_WAITis an indication from the kernel that the socket has been closed on the client side, but is in a waiting to be closed state as far as the kernel is concerned - see https://www.yougetsignal.com/library/time-wait-state-idoy6cuSockets being in
TIME_WAITis not a bug.If that is not the case, then it is an issue. Please show output of netstat as above.
Note
_disconnectis called automatically on object deletion which does socket shutdown and detach. Closing sockets inTIME_WAITis a kernel setting, not client configurable. Default behaviour is kernel closes them after 60sec.tzhuang-crank commentedon Oct 22, 2026
Thanks for the explanation regarding TIME_WAIT. I investigated that possibility and agree that many of the connections I was seeing immediately after the test are normal TIME_WAIT sockets.
I used the following minimal reproducer:
I instrumented the disconnect path and confirmed that for every iteration:
were all executed.
Immediately after the test, ss showed many connections in TIME-WAIT state, which is consistent with your explanation. After sleeping 180 seconds, all of those TIME-WAIT entries disappeared.
However, the process file descriptor count remained elevated:
Many of the leaked descriptors correspond to the same ephemeral local ports that previously appeared in the TIME_WAIT output.
The result that surprised me most was the following experiment.
Current code:
Experimental change:
With that modification:
and the kernel immediately began reusing descriptor numbers (e.g. repeatedly reusing fds 5 and 6), indicating that the descriptors were actually being released.
This makes me wonder whether ownership of the detached file descriptor is expected to be transferred somewhere else after detach() is called. I searched through the disconnect path but wasn't able to find where that detached descriptor would subsequently be closed.
Am I missing a later cleanup path, or is detach() expected to hand ownership to another component? If ownership is intentionally transferred, could you point me to where the detached descriptor is ultimately closed?
For transparency, this response and most debugging approaches were suggested by an LLM, but all instrumentation, code changes, measurements, and observations above were collected from local testing against a real reproducer.
tzhuang-crank commentedon Oct 22, 2026
One additional note: the original motivation behind this investigation was a hot loop creating, using, and destroying (via del) many SSHClient instances. In my application I've since worked around the issue by caching and reusing a single client instance, and recreating it only when a SessionError occurs. That has stabilized FD usage for my workload.
So from a practical standpoint I have a workaround available today. My interest at this point is mostly understanding the ownership/lifetime model around the detached socket and whether my observations indicate a genuine leak or simply a misunderstanding on my part.
It would be nice to have finer control over client lifetime in cases where client reuse is not practical, if such a use case ever arises. It's also entirely possible that I'm misunderstanding some aspect of SSH, libssh2, or ssh2-python's cleanup semantics.