Skip to content

Kafka brokers are not accessible from outside of the cluster because they advertise the InternalIP only #569

Open

Description

@fhennig

Affected version

nightly (and possibly older versions)

Current and expected behavior

In the test an initial connection through a port forward is used to connect to 9092, but then the Kafka process advertises InternalIPs (10.x.x.x) which cannot be reached from the outside. The k8s nodes have InternalIPs and ExternalIPs (running on GKE). The operator should pick the ExternalIP over the InternalIP here

Starting port-forwarding of port 9092
Creating test data
Writing test data
%3|1679320241.591|FAIL|rdkafka#producer-1| [thrd:10.10.0.3:31313/1002]: 10.10.0.3:31313/1002: Connect to ipv4#10.10.0.3:31313 failed: Connection timed out (after 131226ms in state CONNECT)
% ERROR: Local: Broker transport failure: 10.10.0.3:31313/1002: Connect to ipv4#10.10.0.3:31313 failed: Connection timed out (after 131226ms in state CONNECT)
%3|1679320372.664|FAIL|rdkafka#producer-1| [thrd:10.10.0.3:31313/1002]: 10.10.0.3:31313/1002: Connect to ipv4#10.10.0.3:31313 failed: Connection timed out (after 130972ms in state CONNECT, 1 identical error(s) suppressed)
% ERROR: Local: Broker transport failure: 10.10.0.3:31313/1002: Connect to ipv4#10.10.0.3:31313 failed: Connection timed out (after 130972ms in state CONNECT, 1 identical error(s) suppressed)
% Delivery failed for message: Local: Message timed out

Possible solution

The listener operator should fix this (?)

Additional context

Environment

Would you like to work on fixing this bug?

None

Activity

  1. sbernauer commented on Mar 21, 2023

    @sbernauer
    Member

    Should be a pretty easy fix, on the other hand I would also like to implement usage of listener-op in kafka, which might make more sense

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions