[3.15] Backport GH-148874: Make sure that mngr.__exit__() is always called in a with statement (GH-150911)

[3.15] Backport GH-148874: Make sure that mngr.__exit__() is always called in a with statement (GH-150911) #127327