Repository navigation
Description
The conditions that caused this were a bit unusual (tl;dr: for a tool I was working on, it was cheaper to rebuild lots of KeyBindings rather than store them persistently). So it might be a rare bug, but it can turn up in practice.
Description
DynamicKeyBindingssetsself._last_versionto(id(key_bindings), key_bindings._version), which is then returned by_Proxy._versionand used by_MergedKeyBindingsto decide whether the registry is rebuilt.- Since Python only guarantees a unique
idamong objects whose lifetime overlaps, if a priorkey_bindingsis freed it is possible for a laterkey_bindingsto end up with the sameid()and_versionvalue. - The object whose
idwas recorded can be freed shortly afterwards, because_MergedKeyBindingsaccesses_versionand thenbindings, so the callback is called twice. So unless stored elsewhere, the first object is replaced and itsidbecomes available for a laterkey_bindings. - Since
_MergedKeyBindings._update_cacheuses this to decide if the registry should be rebuilt, it is possible for it to see an unchanged version for different objects and skip its rebuild, retaining the bindings it copied from the previous object.
Consequently: under certain conditions, bindings provided by DynamicKeyBindings can become stale.
Steps to reproduce
In each case, I have used prompt_toolkit version 3.0.53.
It is definitely machine-dependent. I can reproduce it on:
- macOS 26.6.2 with Python 3.15.2, and
- Debian 13.6 with Python 3.13.5.
But I could only rarely reproduce it on Gentoo 2.18 with any of Python 3.12.16, 3.13.18 or 3.14.10, so it seems to depend on the allocator.
The first example below passes the DynamicKeyBindings directly to an Application which merges the bindings internally. In the second, the internal behaviour is reproduced so merge_key_bindings is called manually.
Example 1
An interactive (and slightly contrived) reproduction: this counting app displays a few counters in rows. Key j hops through the list of counters, and i increments the active counter. Often, pressing j then i will increment the previously selected counter. This is here just to show that it can happen in a real application and not just in the artificial automated case.
Show code
from prompt_toolkit.application import Application
from prompt_toolkit.key_binding import DynamicKeyBindings, KeyBindings
from prompt_toolkit.layout import FormattedTextControl, Window, Layout
counts = [0] * 6
selected = 0
def hop(_):
global selected
selected = (selected + 1) % len(counts)
def render():
rows = [f"{'>' if i == selected else ' '} Counter {i}: {c}" for i, c in enumerate(counts)]
return "\n".join(rows + ["", "j: next i: increment q: quit"])
def bindings():
kb = KeyBindings()
target = selected
kb.add("j")(hop)
kb.add("i")(lambda _: counts.__setitem__(target, counts[target] + 1))
kb.add("q")(lambda e: e.app.exit())
return kb
Application(layout=Layout(Window(FormattedTextControl(render))),
key_bindings=DynamicKeyBindings(bindings)).run()Example 2
A non-interactive/automated version that mimics the internal behaviour without actually running an app.
from prompt_toolkit.key_binding import DynamicKeyBindings, KeyBindings, merge_key_bindings
counts = [0] * 6
selected = 0
track = []
def hop(_):
global selected
selected = (selected + 1) % len(counts)
def bindings():
kb = KeyBindings()
target = selected
kb.add("j")(hop)
def inc(_, target=target):
if target != selected:
track.append((target, selected))
kb.add("i")(inc)
return kb
n = 10000
# merge_key_bindings to enforce a cache comparison
keys = merge_key_bindings([DynamicKeyBindings(bindings)])
for _ in range(n):
j_key = keys.get_bindings_for_keys(("j",))
keys.get_bindings_starting_with_keys(("j",)) # reproducing internal steps that an Application does
j_key[-1].handler(None)
i_key = keys.get_bindings_for_keys(("i",))
keys.get_bindings_starting_with_keys(("i",))
i_key[-1].handler(None)
print(f"{len(track)} / {n} stale lookups")On macOS and Debian, this consistently outputs 9998 / 10000 stale lookups. On Gentoo it was much rarer (usually 0, but once I saw a 3).
Suggested fix
Instead of using id to determine changes, keep a counter in DynamicKeyBindings that increases whenever the callback returns a different object, and report (counter, key_bindings._version). The objects are compared directly using is rather than by comparing their id.
A similar approach may have been intended in the past, because there are remnants of it already existing in the code. I have a small patch completing this implementation and will open a PR with it shortly.
With this patch, the second example script reports 0 / 10000 stale lookups on all machines I tried it on.
Activity