Skip to content

fix: Strip breadcrumb metadata on OOM to prevent silent event loss#2150

Open
spawnia wants to merge 8 commits into
getsentry:masterfrom
mll-lab:fix-oom-breadcrumb-serialization
Open

fix: Strip breadcrumb metadata on OOM to prevent silent event loss#2150
spawnia wants to merge 8 commits into
getsentry:masterfrom
mll-lab:fix-oom-breadcrumb-serialization

Conversation

@spawnia

@spawnia spawnia commented Jul 7, 2026

Copy link
Copy Markdown

Fixes getsentry/sentry-laravel#909

When an out-of-memory error occurs with large breadcrumbs (e.g. 100 SQL queries with ~100KB bindings each), the event is silently lost. Serialization needs ~15MB of working memory but only 5MB headroom is available after ErrorHandler raises the limit. The secondary OOM during json_encode is caught by invokeListeners() and discarded.

This fix detects OOM in FatalErrorListenerIntegration and strips breadcrumb metadata before capture — freeing the bulk of the memory while preserving the full breadcrumb trail (type, category, timestamp, message) for debugging.

Design decisions

Why strip metadata instead of clearing all breadcrumbs? The breadcrumb timeline (which queries ran, in what order) is valuable for diagnosing OOM. The metadata (SQL bindings, HTTP bodies) is what carries the bulk (~99% of size) and is rarely needed for OOM diagnosis.

Why in the integration, not ErrorHandler or PayloadSerializer? The integration already has access to the exception message for OOM detection. Putting it in ErrorHandler would couple the low-level handler to Hub/Scope. Catching OOM in the serializer is unreliable since PHP's nested try/catch behavior during OOM is fragile.

Why modify the scope directly instead of using an event processor? The scope's breadcrumb objects hold references to the large metadata strings. These must be freed from PHP's heap before serialization to make room. An event processor runs after applyToEvent copies breadcrumbs to the event — by then the scope still holds the originals in memory and the headroom is already consumed.

Why add Scope::getBreadcrumbs()? We need to read the breadcrumbs to build stripped copies before clearing. The scope had no getter — addBreadcrumb and clearBreadcrumbs existed but you couldn't iterate the collection.

🤖 Generated with Claude Code

spawnia added 5 commits July 7, 2026 11:20
… serialization

When an out-of-memory error occurs with large breadcrumbs (e.g. SQL
queries with bindings), serializing the event requires more working
memory than the 5MB headroom provided by ErrorHandler. This causes a
secondary OOM during json_encode in PayloadSerializer, which is
silently caught and discards the entire event.

Detect OOM in FatalErrorListenerIntegration and replace breadcrumbs
with metadata-stripped copies before capture. This preserves the full
breadcrumb trail (type, category, level, timestamp, message) while
freeing the bulky data fields that cause the serialization to exceed
available memory.

Also adds Scope::getBreadcrumbs() to enable reading breadcrumbs for
transformation without introducing an event processor that would run
after the large breadcrumbs are already copied to the event.

Fixes getsentry/sentry-laravel#909

🤖 Generated with Claude Code
The re-add loop after stripping metadata used addBreadcrumb's default
cap of 100, silently truncating breadcrumbs when the user configured a
higher limit. Pass the actual count so all stripped breadcrumbs survive.

🤖 Generated with Claude Code
PHP 8.5 appends a stack trace to the fatal error output, requiring a
separate EXPECTF block. Also remove unused $serialized variable from
the php84 variant.

🤖 Generated with Claude Code
The FatalErrorException message has an "Error: " prefix, so the anchor
from ErrorHandler's pattern would not match. No captures are needed
since this is a boolean check.

🤖 Generated with Claude Code
@spawnia spawnia marked this pull request as ready for review July 7, 2026 10:15

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Want reviews to match your repository better? Bugbot Learning can learn team-specific rules from PR activity. A team admin can enable Learning in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit c30b256. Configure here.

spawnia and others added 3 commits July 7, 2026 12:23
The php84/ OOM tests used >= 80400, excluding PHP 8.4 itself. The stack
trace output change only happened in 8.5, so these tests should run on
8.4 as well — matching the tests/phpt/php84/ convention of >= 80500.

getsentry#2150 (comment)

🤖 Generated with Claude Code
Moves error_reporting() before require autoload.php so that
OpenTelemetry's implicit-nullable deprecation notices on PHP 8.4/8.5
lowest deps don't pollute the PHPT expected output.

🤖 Generated with Claude Code
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

OOM events silently lost when breadcrumb serialization exceeds 5 MB headroom

1 participant