LoadbalancerPool should remove targets which it fails to connect

[pckeyan]

I am trying to add the following nodes to the Load balancer as below:

LoadbalanceTarget target9000 = LoadbalanceTarget.from("9000", TcpClientTransport.create("localhost",9000));
        LoadbalanceTarget target9001 = LoadbalanceTarget.from("9001", TcpClientTransport.create("localhost",9001));
        LoadbalanceTarget target9002 = LoadbalanceTarget.from("9002", TcpClientTransport.create("localhost",9002));
        ArrayList<LoadbalanceTarget> serverFarm = new ArrayList<>();
        serverFarm.add(target9000);
        serverFarm.add(target9001);
        serverFarm.add(target9002);
        Flux<List<LoadbalanceTarget>> producer = Flux.fromIterable(serverFarm).collectList().flux();
        RSocketClient rSocketClient =
                LoadbalanceRSocketClient.builder(producer).roundRobinLoadbalanceStrategy().build();

While in the process of the application start up, one of the node failed and not available.

Expected Behavior

Load Balancer should gracefully not use the node that is not reachable.

Actual Behavior

Client application fails to start with following exception:

io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection refused: localhost/127.0.0.1:9002
Caused by: java.net.ConnectException: Connection refused
	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
	at io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:330)
	at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:334)
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:702)
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650)
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576)
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
	at java.lang.Thread.run(Thread.java:748)

Possible Solution

Loadbalancer should gracefully not send the request to the unavailable node and application should continue to process requests in other nodes.

Your Environment

   <parent>
        <groupId>org.springframework.boot</groupId>
          <artifactId>spring-boot-starter-parent</artifactId>
        <version>2.3.5.RELEASE</version>
        <relativePath/> <!-- lookup parent from repository -->
    </parent>
    <dependency>
            <groupId>io.projectreactor</groupId>
            <artifactId>reactor-core</artifactId>
            <version>3.4.0</version>
    </dependency>

• RSocket version(s) used: 1.1.0

• Other relevant libraries versions (eg. netty, ...):

• Platform (eg. JVM version (javar -version) or Node version (node --version)): Java 8

• OS and version (eg uname -a): Mac

[rstoyanchev]

Superseded by #982.