fix: guard TelemetryEventBuffer against re-entrant mutex acquisition#2964
Merged
Conversation
MetricEventBuffer uses a non-reentrant Mutex. When HTTP instrumentation (e.g. yabeda-http_requests, yabeda-rails) observes the outgoing Net::HTTP call made by HTTPTransport#send_data, it triggers Sentry::Yabeda::Adapter which calls Sentry.metrics.* -> MetricEventBuffer#add_item on the same thread that already holds the buffer mutex, raising: ThreadError: deadlock; recursive locking Fix: set a thread-local flag (SENTRY_SENDING_KEY) in HTTPTransport for the duration of the do_request call and check it in all Adapter#perform_*! methods, silently skipping metric forwarding for Sentry's own HTTP sends. Fixes #2963 Co-Authored-By: junior <noreply@example.com> Co-authored-by: neel <neel.shah@sentry.io>
Any transport instrumentation that calls Sentry.metrics.* from within transport#send_data re-enters MetricEventBuffer#add_item on the same thread that already holds @Mutex inside send_items, raising: ThreadError: deadlock; recursive locking This includes any non-Yabeda code such as custom transports or other HTTP instrumentation (see getsentry/repro#46 for a minimal repro using a ReentrantTransport that calls Sentry.metrics directly from send_data). Fix: check @mutex.owned? before acquiring in add_item and silently drop the item when the current thread already holds the lock. This is the defensive fix at the correct abstraction level and protects all buffers (MetricEventBuffer, LogEventBuffer) from all callers. The sentry-yabeda Adapter guard (HTTPTransport.sending?) is retained as defense-in-depth and avoids constructing a MetricEvent that will be dropped, but the buffer-level fix is the load-bearing protection. Refs #2963 Refs getsentry/repro#46 Co-Authored-By: junior <noreply@example.com> Co-authored-by: johdax <johannes.daxboeck@sentry.io>
Replace instance_double with []: nil keyword syntax (invalid in Ruby 2.7 parser) with build_fake_response, consistent with the rest of the spec. Co-Authored-By: junior <noreply@example.com> Co-authored-by: johdax <johannes.daxboeck@sentry.io>
… guard Remove the thread-local flag layer (HTTPTransport.sending?, with_sentry_sending, Yabeda adapter sentry_http_sending? checks) in favor of the @mutex.owned? guard in TelemetryEventBuffer alone. This is simpler, more general, and eliminates cross-gem coupling. Also add the guard to flush which had the same exposure. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
sl0thentr0py
approved these changes
Jun 8, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
MetricEventBuffer(andLogEventBuffer) use a non-reentrantMutex. When transport instrumentation callsSentry.metrics.*from insidesend_items— while the mutex is already held — the same thread tries to acquire it again, raising:The re-entrant call can come from any code that fires during an outgoing HTTP request:
yabeda-http_requests,yabeda-rails)Sentry.metrics.*duringsend_data(see getsentry/repro#46)The repeated deadlock prevents the buffer from draining, causing the steady memory growth reported in #2963.
Fix —
@mutex.owned?guard inTelemetryEventBufferA single guard in the base class protects all buffers (metrics, logs) against all re-entrant callers, regardless of origin:
Mutex#owned?returnstrueonly when the same thread already holds the lock — the exact condition that would deadlock. Items from re-entrant calls are silently dropped. Normal concurrent access from other threads is unaffected (they block onsynchronizeas before).Both
add_itemandflushare guarded because both acquire@mutexand callsend_items.Why not also guard at the integration layer?
An earlier iteration added a thread-local flag in
HTTPTransport(sending?) and checks inSentry::Yabeda::Adapterto skip metric forwarding during transport sends. This was removed because:@mutex.owned?guard is sufficient — it catches the deadlock at the exact point it would occurCall chain (with fix applied)
Changes
sentry-ruby/lib/sentry/telemetry_event_buffer.rb@mutex.owned?guard inadd_itemandflushsentry-ruby/spec/support/shared_examples_for_telemetry_event_buffers.rbFixes #2963
Refs getsentry/repro#46