Repository navigation
Deadlock on executing commands from binding.b in Rails app #725
Description
Your environment
❯ ruby -v
ruby 3.1.2p20 (2022-04-12 revision 4491bb740a) [x86_64-darwin20]
❯ rdbg -v
rdbg 1.6.1Describe the bug
When using binding.b in our Rails app — in various places: code run from test suites, code run from serving web requests — the debug REPL will hang.
Here is an example session entering the debugger from unit tests:
.[5, 14] in ~/src/debug_hang/app/models/concerns/debug_hang/sample_concern.rb
5| def respond_to_missing?(method, include_private = false)
6| method.to_s.start_with?("test_") || super
7| end
8|
9| def method_missing(method, *args, &block)
=> 10| binding.b
11|
12| if method.to_s =~ /^test_/
13| puts "Running test: #{method}"
14| super
=>#0 DebugHang::SampleConcern#method_missing(method=:hang_the_debugger, args=[], block=nil) at ~/src/debug_hang/app/models/concerns/debug_hang/sample_concern.rb:10
#1 block in <class:ExampleTest> at ~/src/debug_hang/test/models/example_test.rb:5
# and 25 frames (use `bt' command for all frames)
(ruby) false == true
false
(ruby) false == true
["DEBUGGER Exception: /Users/ian/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/debug-1.6.1/lib/debug/thread_client.rb:1108",
#<fatal: No live threads left. Deadlock?
20 threads, 20 sleeps current:0x00007f8590c458b0 main thread:0x00007f85f6c06d30
* #<Thread:0x0000000105c30ce8 sleep_forever>
rb_thread_t:0x00007f85f6c06d30 native:0x0000000106beae00 int:0
To Reproduce
git clone git@github.com:ianterrell/debug_hang.git
cd debug_hang
bundle install
rails db:create db:migrate
rails tFrom debugger, repeatedly enter false == true.
Expected behavior
No deadlock.
Additional context
- This looks like it could be related to binding.b(do: ...) hangs Rails with puma web server #248
Activity
yshmarov commentedon Jul 21, 2022
I'm experiencing the same issue right at this moment. When running some console commands the binding just dies out
I tried downgrading the gem to
gem 'debug', '~> 1.5.0', '<= 1.6.0'=> no deadlockianterrell commentedon Jul 21, 2022
@yshmarov Thanks for noting the downgrade suggestion. My reproduction case succeeds on 1.5.0. Running against a local copy of
debugand bisecting I get the first bad commit as 00ec229. That's a threading related commit, so it seems plausible.st0012 commentedon Jul 21, 2022
I haven't located the cause, but I found that we can reproduce it just by using these:
And it seems that there's a thread created before things hang:
It's the thread's suspension caused the hang and deadlock.
But I'm still figuring out:
@yshmarov Can you briefly describe the way you encountered this problem? Is it also in minitest?
Update: repro steps can be further reduced to just type
Timeout.send :create_timeout_threadwhen entering the console.ianterrell commentedon Jul 21, 2022
@st0012 I've noticed the issue is happening if I use my up key to go through the history. That appears to be what triggers queuing the
:thread_beginevent.st0012 commentedon Jul 21, 2022
I've created #727 for this. Can you give my branch a try and let me know if the issue still persists or something else occurs? You can do it with:
Thanks
ianterrell commentedon Jul 22, 2022
@st0012 So far that fix looks good! 🎉
It fixes my reproduction case and the similar one in our actual codebase.
My colleague had a different reproduction case that occurred outside of MiniTest while simply running the app. We think that one may have been occurring even on the 1.5.0 release. But so far that appears fixed on this branch, too.
st0012 commentedon Jul 22, 2022
Thank you for the feedback ❤️
Petercopter commentedon Jul 29, 2022
@st0012 Works for me as well. Doing a lot of debugging right now and it was driving me crazy! Thanks 👍