Skip to content

Guard against f_lasti == -1 when detecting exception returns#266

Merged
cool-RR merged 2 commits into
cool-RR:masterfrom
koriyoshi2041:fix/guard-negative-f-lasti
Jun 8, 2026
Merged

Guard against f_lasti == -1 when detecting exception returns#266
cool-RR merged 2 commits into
cool-RR:masterfrom
koriyoshi2041:fix/guard-negative-f-lasti

Conversation

@koriyoshi2041

Copy link
Copy Markdown
Contributor

Fixes #260.

To distinguish a normal return event from one caused by an exception (both arrive as event == 'return' with arg is None), the tracer inspects the last executed opcode:

code_byte = frame.f_code.co_code[frame.f_lasti]
...
ended_by_exception = (event == 'return' and arg is None
                      and opcode.opname[code_byte] not in RETURN_OPCODES)

frame.f_lasti can be -1 (for example a frame that hasn't executed an instruction yet). co_code[-1] then reads the last byte of the bytecode — an unrelated opcode — and the result can flip ended_by_exception to True, so a normal return is printed as Call ended by exception.

This extracts the check into call_ended_by_exception(frame, event, arg) and returns False when f_lasti < 0, so the wrong opcode is never read.

Added tests/test_ended_by_exception.py with mock frames: f_lasti == -1 is treated as a normal return; a return opcode → normal return; a non-return opcode → ended by exception; non-return events are never flagged. The f_lasti == -1 case fails without the guard and passes with it; existing tracer tests still pass.

The tracer reads co_code[frame.f_lasti] to decide whether a 'return' event was
caused by an exception. f_lasti can be -1 (e.g. a frame that hasn't executed
an instruction), and co_code[-1] reads the last byte -- the wrong opcode --
which can misreport a normal return as "Call ended by exception". Extract the
check into call_ended_by_exception() and return False when f_lasti < 0.

@cool-RR cool-RR left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @koriyoshi2041, good catch and a clean fix. I went and verified it carefully and it holds up.

The bug is real on CPython 3.8 to 3.10 (and latent on 3.11+).

The repro in #260 is mock-only, so I checked whether a real return event ever actually carries f_lasti == -1. Across normal returns, exceptions, and a pile of generator scenarios:

  • 3.11 / 3.12 / 3.13: never happens. Purely latent.
  • 3.8 / 3.9 / 3.10: it does. Closing (or throw()-ing into) an unstarted generator fires a return event whose frame has f_lasti == -1, which is exactly the case where co_code[-1] reads an unrelated byte.

And it surfaces through our own depth>=2 path. On 3.8, master vs this PR for a plain g = gen(5); g.close():

master                              this PR
  exception 12 def gen(n):            exception 12 def gen(n):
  Call ended by exception      -->    return    12 def gen(n):
                                       Return value:.. None

master's "exception" verdict there is just luck of the last bytecode byte, not detection, so I'm glad to see it go.

Verification

  • Full suite passes on 3.8, 3.12, 3.13 (plus the 4 new unit tests).
  • Behavior is identical to master except the f_lasti < 0 case, which is the fix. It also can't hide a genuine error: with nothing executed in the frame the only way out is an injected exception, which still gets its own exception event.
  • Bonus: moving the opcode read behind the event == 'return' check skips it on every line event, which is the hot path.

How others handle it, for reference

  • python-hunter guards the same way before indexing: len(co_code) > f_lasti >= 0. Same instinct as this PR.
  • snoop has the identical unguarded co_code[f_lasti] and nobody's hit it, which matches the "latent on modern CPython" finding.
  • stdlib bdb/pdb sidestep it entirely by routing on the event string and never reading opcodes.

Two small things

  1. The new tests are unit-only (mocked frames). Fair, since the bug isn't reachable on CI Python, but I'd like a real integration test alongside them (below). It passes on all versions and would have failed on master for 3.8 to 3.10.
  2. tests/test_ended_by_exception.py imports unittest.mock, which is Py3-only, while setup.py still lists 2.7. That 2.7 classifier is vestigial and I'll drop it; just flagging so the test file isn't what trips over it.

Suggested integration test (verified: passes on this PR for 3.8/3.10/3.12/3.13, fails on master for 3.8/3.10):

def test_unstarted_generator_close_is_not_an_exception():
    # Regression for #260. On CPython 3.8-3.10, closing an unstarted generator
    # fires a 'return' event whose frame has f_lasti == -1; the old code read
    # co_code[-1] (an unrelated opcode) and mislabeled this normal close as
    # "Call ended by exception". Passes on all versions; only catches the bug
    # on 3.8-3.10, where it is reachable.
    string_io = io.StringIO()

    def gen(n):
        for i in range(n):
            yield i

    @pysnooper.snoop(string_io, depth=2, color=False)
    def main():
        g = gen(5)
        g.close()
        return 42

    assert main() == 42
    output = string_io.getvalue()
    assert 'Call ended by exception' not in output
    assert 'Return value:.. 42' in output

(needs import io and import pysnooper at the top of the file.)

Add that and I'm happy to merge. Thanks again!

@koriyoshi2041

Copy link
Copy Markdown
Contributor Author

Added the integration regression test you suggested and pushed it as 393eb37.

Checked locally with:

uv run --with pytest python -m pytest tests/test_ended_by_exception.py -q
# 5 passed

@cool-RR cool-RR merged commit 19f39a0 into cool-RR:master Jun 8, 2026
@cool-RR

cool-RR commented Jun 8, 2026

Copy link
Copy Markdown
Owner

Good job, thank you.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Indexing frame.f_code.co_code at frame.f_lasti can use -1 (invalid) leading to wrong opcode or IndexError

2 participants