Skip to content

Fix lock-order inversion between the runtime lock and the selector table lock#394

Merged
davidchisnall merged 1 commit into
gnustep:masterfrom
DTW-Thalion:fix/selector-registration-lock-order
Jul 1, 2026
Merged

Fix lock-order inversion between the runtime lock and the selector table lock#394
davidchisnall merged 1 commit into
gnustep:masterfrom
DTW-Thalion:fix/selector-registration-lock-order

Conversation

@DTW-Thalion

Copy link
Copy Markdown
Contributor

Fixes #391.

Registering a new selector resizes the dtables, and objc_resize_dtables takes the runtime lock (dtable.c:618) while the selector table lock is already held (objc_register_selector_copyregister_selector_locked). The class loader does the opposite — __objc_load holds the runtime lock (loader.c:186) and then registers selectors, taking the selector table lock (loader.c:217selector_lookup). That's the runtime_mutexselector_table_lock cycle, reachable when a concurrent dlopen of an Objective-C module races a selector registration that triggers a dtable resize.

This holds the runtime lock for the duration of registration, acquiring it before the selector table lock in both registration entry points, so every path takes the two locks in the same runtime → selector order. It also keeps the new selector's index (the selector_list size) and the matching dtable resize atomic with respect to other resizes, which registration relies on. The runtime lock is recursive, so the nested acquisition inside objc_resize_dtables is unaffected, and the already-registered fast paths still take neither lock.

On the dedicated-lock idea from #391

I started down the "new lock that protects dtable resizes" path, but objc_resize_dtables walks the class table (class_table_next), and class_table_insert / class_table_next have no locking of their own — they rely on the caller holding the runtime lock. So a resize can't move off the runtime lock without also moving all class-table protection onto the new lock, which cascades into decomposing the big runtime lock more broadly (the audit you mentioned). The lock that protects dtable resizes today is the runtime lock; this change does the minimal version of that suggestion — hold it for the duration of registration — using the lock that already guards the resize. Happy to pursue the finer-grained decomposition separately if you'd prefer.

Validation

  • TSan, selector-stress harness (concurrent sel_registerName + class load at startup): clean master reports the inversion deterministically (2 reports); with this change, 0. Harness runs to completion both ways.
  • Functional: full ctest suite 194/194 pass, including ManyManySelectors.

The remaining TSan data races on the selector_list size are a separate, pre-existing unsynchronized-read finding and are not addressed here.

…ration

Registering a new selector resizes the dtables, and objc_resize_dtables
takes the runtime lock while the selector table lock is already held.  The
class loader (__objc_load) does the opposite: it holds the runtime lock and
then registers selectors, taking the selector table lock.  These two orders
form a lock-order inversion between the runtime mutex and the selector table
lock (gnustep#391), reachable when a dynamic class load races with a
selector registration that triggers a dtable resize.

Acquire the runtime lock before the selector table lock in the two
registration entry points so both paths take the locks in the same
runtime-before-table order.  This also keeps the new selector's index (the
selector_list size) and the matching dtable resize atomic with respect to
other resizes, which registration relies on.  The runtime lock is recursive,
so the nested acquisition inside objc_resize_dtables is unaffected, and the
already-registered fast paths still take neither lock.
@davidchisnall davidchisnall merged commit 89722f9 into gnustep:master Jul 1, 2026
87 of 91 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

Lock-order inversion between selector_table_lock and runtime_mutex (TSan potential deadlock)

2 participants