Skip to content

Flaky test: ClientTest.testCloseClient #167

Open

Description

@BewareMyPower

example failure

FAILED TESTS (5/418):
     743 ms: ./pulsar-tests ClientTest.testReferenceCount (try #1)
   11653 ms: ./pulsar-tests ClientTest.testCloseClient (try #1)
    9692 ms: ./pulsar-tests ClientTest.testCloseClient (try #2)
    8242 ms: ./pulsar-tests ClientTest.testCloseClient (try #3)
    8126 ms: ./pulsar-tests ClientTest.testCloseClient (try #4)
// ...
2022-12-29T01:22:46.5133298Z 2022-12-29 01:22:46.253 INFO  [140291349905088] ConnectionPool:97 | Created connection for pulsar://localhost:6650
2022-12-29T01:22:46.5133649Z 2022-12-29 01:22:46.253 INFO  [140291346781760] ClientConnection:387 | [[::1]:60292 -> [::1]:6650] Connected to broker
2022-12-29T01:22:46.5133967Z 2022-12-29 01:22:46.253 INFO  [140291349905088] ClientImpl:516 | Closing Pulsar client with 0 producers and 0 consumers
2022-12-29T01:22:46.5134331Z 2022-12-29 01:22:46.254 INFO  [140291330000448] ClientConnection:1602 | [[::1]:60292 -> [::1]:6650] Connection disconnected
2022-12-29T01:22:46.5134974Z 2022-12-29 01:22:46.254 ERROR [140291346781760] ClientImpl:183 | Error Checking/Getting Partition Metadata while creating producer on persistent://public/default/client-test-close-client-1672276958 -- NotConnected
2022-12-29T01:22:46.5135316Z 2022-12-29 01:22:46.254 INFO  [140291330000448] ClientConnection:268 | [[::1]:60292 -> [::1]:6650] Destroyed connection
2022-12-29T01:22:46.5135700Z 2022-12-29 01:22:46.255 INFO  [140291349905088] ClientConnection:190 | [<none> -> pulsar://localhost:6650] Create ClientConnection, timeout=10000
2022-12-29T01:22:46.5136036Z 2022-12-29 01:22:46.255 INFO  [140291349905088] ConnectionPool:97 | Created connection for pulsar://localhost:6650
2022-12-29T01:22:46.5136486Z 2022-12-29 01:22:46.255 INFO  [140291330000448] ClientConnection:387 | [[::1]:60300 -> [::1]:6650] Connected to broker
2022-12-29T01:22:46.5136808Z 2022-12-29 01:22:46.255 INFO  [140291349905088] ClientImpl:516 | Closing Pulsar client with 0 producers and 0 consumers
2022-12-29T01:22:46.5137078Z [208/418] ClientTest.testCloseClient returned with exit code -11 (8126 ms)

It can be easily reproduced in local env by:

./tests/pulsar-tests --gtest_filter='ClientTest.testCloseClient' --gtest_repeat=20

Activity

  1. self-assigned this
    on Jan 5, 2023
  2. BewareMyPower commented on Jul 4, 2023

    @BewareMyPower
    ContributorAuthor

    This test is still very flaky on macOS.

    I tested tag v3.2.0.

    * thread #4, stop reason = EXC_BAD_ACCESS (code=1, address=0x98)
        frame #0: 0x0000000100070d44 pulsar-tests`boost::asio::detail::kqueue_reactor::deregister_descriptor(this=0x0000000103804820, descriptor=8, descriptor_data=0x0000600002608070, closing=true) at kqueue_reactor.ipp:336:25
       333 	
       334 	  mutex::scoped_lock descriptor_lock(descriptor_data->mutex_);
       335 	
    -> 336 	  if (!descriptor_data->shutdown_)
       337 	  {
       338 	    if (closing)
       339 	    {
    Target 0: (pulsar-tests) stopped.
    
    (lldb) bt
    * thread #4, stop reason = EXC_BAD_ACCESS (code=1, address=0x98)
      * frame #0: 0x0000000100070d44 pulsar-tests`boost::asio::detail::kqueue_reactor::deregister_descriptor(this=0x0000000103804820, descriptor=8, descriptor_data=0x0000600002608070, closing=true) at kqueue_reactor.ipp:336:25
        frame #1: 0x00000001043126a0 libpulsar.dylib`boost::asio::detail::reactive_socket_service_base::close(this=0x0000600002114078, impl=0x0000600002608068, ec=0x000000016ff9dca0) at reactive_socket_service_base.ipp:110:14
        frame #2: 0x00000001042ca264 libpulsar.dylib`boost::asio::basic_socket<boost::asio::ip::tcp, boost::asio::any_io_executor>::close(this=0x0000600002608060, ec=0x000000016ff9dca0) at basic_socket.hpp:547:25
        frame #3: 0x00000001042d73c0 libpulsar.dylib`pulsar::ClientConnection::closeSocket(this=0x0000000106809400) at ClientConnection.cc:1334:18
        frame #4: 0x00000001042c49ec libpulsar.dylib`pulsar::ClientConnection::close(this=0x0000000106809400, result=ResultDisconnected) at ClientConnection.cc:1163:5
        frame #5: 0x00000001043eb718 libpulsar.dylib`pulsar::ConnectionPool::close(this=0x0000000103804308) at ConnectionPool.cc:56:22
        frame #6: 0x0000000104362320 libpulsar.dylib`pulsar::ClientImpl::shutdown(this=0x00000001038041a8) at ClientImpl.cc:703:16
        frame #7: 0x00000001043d5c5c libpulsar.dylib`pulsar::ClientImpl::handleClose(this=0x0000600001708408)>)::$_4::operator()() const at ClientImpl.cc:668:13
        frame #8: 0x00000001043d5c00 libpulsar.dylib`decltype(__f=0x0000600001708408)>)::$_4>()()) std::__1::__invoke[abi:v15006]<pulsar::ClientImpl::handleClose(pulsar::Result, std::__1::shared_ptr<int>, std::__1::function<void (pulsar::Result)>)::$_4>(pulsar::ClientImpl::handleClose(pulsar::Result, std::__1::shared_ptr<int>, std::__1::function<void (pulsar::Result)>)::$_4&&) at invoke.h:394:23
        frame #9: 0x00000001043d5b9c libpulsar.dylib`void std::__1::__thread_execute[abi:v15006]<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, pulsar::ClientImpl::handleClose(pulsar::Result, std::__1::shared_ptr<int>, std::__1::function<void (pulsar::Result)>)::$_4>(__t=size=2, (null)=__tuple_indices<> @ 0x000000016ff9ef7f)>)::$_4>&, std::__1::__tuple_indices<>) at thread:290:5
        frame #10: 0x00000001043d553c libpulsar.dylib`void* std::__1::__thread_proxy[abi:v15006]<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, pulsar::ClientImpl::handleClose(pulsar::Result, std::__1::shared_ptr<int>, std::__1::function<void (pulsar::Result)>)::$_4> >(__vp=0x0000600001708400) at thread:301:5
        frame #11: 0x0000000188213fa8 libsystem_pthread.dylib`_pthread_start + 148
    
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions