Skip to content

DynamicKeyBindings._version can collide for distinct KeyBindings instances, causing stale bindings #2165

Open

Description

@hailsaturn

The conditions that caused this were a bit unusual (tl;dr: for a tool I was working on, it was cheaper to rebuild lots of KeyBindings rather than store them persistently). So it might be a rare bug, but it can turn up in practice.

Description

  • DynamicKeyBindings sets self._last_version to (id(key_bindings), key_bindings._version), which is then returned by _Proxy._version and used by _MergedKeyBindings to decide whether the registry is rebuilt.
  • Since Python only guarantees a unique id among objects whose lifetime overlaps, if a prior key_bindings is freed it is possible for a later key_bindings to end up with the same id() and _version value.
  • The object whose id was recorded can be freed shortly afterwards, because _MergedKeyBindings accesses _version and then bindings, so the callback is called twice. So unless stored elsewhere, the first object is replaced and its id becomes available for a later key_bindings.
  • Since _MergedKeyBindings._update_cache uses this to decide if the registry should be rebuilt, it is possible for it to see an unchanged version for different objects and skip its rebuild, retaining the bindings it copied from the previous object.

Consequently: under certain conditions, bindings provided by DynamicKeyBindings can become stale.

Steps to reproduce

In each case, I have used prompt_toolkit version 3.0.53.
It is definitely machine-dependent. I can reproduce it on:

  • macOS 26.6.2 with Python 3.15.2, and
  • Debian 13.6 with Python 3.13.5.

But I could only rarely reproduce it on Gentoo 2.18 with any of Python 3.12.16, 3.13.18 or 3.14.10, so it seems to depend on the allocator.

The first example below passes the DynamicKeyBindings directly to an Application which merges the bindings internally. In the second, the internal behaviour is reproduced so merge_key_bindings is called manually.

Example 1

An interactive (and slightly contrived) reproduction: this counting app displays a few counters in rows. Key j hops through the list of counters, and i increments the active counter. Often, pressing j then i will increment the previously selected counter. This is here just to show that it can happen in a real application and not just in the artificial automated case.

Show code
from prompt_toolkit.application import Application
from prompt_toolkit.key_binding import DynamicKeyBindings, KeyBindings
from prompt_toolkit.layout import FormattedTextControl, Window, Layout

counts = [0] * 6
selected = 0

def hop(_):
    global selected
    selected = (selected + 1) % len(counts)

def render():
    rows = [f"{'>' if i == selected else ' '} Counter {i}: {c}" for i, c in enumerate(counts)]
    return "\n".join(rows + ["", "j: next   i: increment   q: quit"])

def bindings():
    kb = KeyBindings()
    target = selected 
    kb.add("j")(hop)
    kb.add("i")(lambda _: counts.__setitem__(target, counts[target] + 1))
    kb.add("q")(lambda e: e.app.exit())
    return kb

Application(layout=Layout(Window(FormattedTextControl(render))),
            key_bindings=DynamicKeyBindings(bindings)).run()

Example 2

A non-interactive/automated version that mimics the internal behaviour without actually running an app.

from prompt_toolkit.key_binding import DynamicKeyBindings, KeyBindings, merge_key_bindings

counts = [0] * 6
selected = 0
track = []

def hop(_):
    global selected
    selected = (selected + 1) % len(counts)

def bindings():
    kb = KeyBindings()
    target = selected 
    kb.add("j")(hop)
    def inc(_, target=target):
        if target != selected:
            track.append((target, selected))

    kb.add("i")(inc)
    return kb

n = 10000
# merge_key_bindings to enforce a cache comparison
keys = merge_key_bindings([DynamicKeyBindings(bindings)])
for _ in range(n):
    j_key = keys.get_bindings_for_keys(("j",))
    keys.get_bindings_starting_with_keys(("j",)) # reproducing internal steps that an Application does
    j_key[-1].handler(None)
    i_key = keys.get_bindings_for_keys(("i",))
    keys.get_bindings_starting_with_keys(("i",))
    i_key[-1].handler(None)

print(f"{len(track)} / {n} stale lookups")

On macOS and Debian, this consistently outputs 9998 / 10000 stale lookups. On Gentoo it was much rarer (usually 0, but once I saw a 3).

Suggested fix

Instead of using id to determine changes, keep a counter in DynamicKeyBindings that increases whenever the callback returns a different object, and report (counter, key_bindings._version). The objects are compared directly using is rather than by comparing their id.

A similar approach may have been intended in the past, because there are remnants of it already existing in the code. I have a small patch completing this implementation and will open a PR with it shortly.

With this patch, the second example script reports 0 / 10000 stale lookups on all machines I tried it on.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions