PYTHON-5737 - BSON encoding/decoding performance improvements#2715
Open
NoahStapp wants to merge 3 commits intomongodb:masterfrom
Open
PYTHON-5737 - BSON encoding/decoding performance improvements#2715NoahStapp wants to merge 3 commits intomongodb:masterfrom
NoahStapp wants to merge 3 commits intomongodb:masterfrom
Conversation
There was a problem hiding this comment.
Pull request overview
This PR implements three performance optimizations to the BSON C extension to improve encode/decode performance by up to ~25% on Python 3.14, bringing it close to Python 3.11 performance levels.
Changes:
- Skip expensive
_type_markerattribute lookups for common built-in types that are known not to have this attribute - Use direct
PyDict_New()API instead of genericPyObject_CallObject()when document_class is dict - Use direct
PyDict_SetItem()API instead of genericPyObject_SetItem()when document_class is dict
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| bson/_cbsonmodule.h | Add is_dict_class field to codec_options_t struct to track when document_class is dict |
| bson/_cbsonmodule.c | Implement three performance optimizations: skip _type_marker lookup for built-in types, use PyDict_New() and PyDict_SetItem() when possible |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
ShaneHarvey
requested changes
Feb 26, 2026
| /* Built-in types don't have _type_marker, skip the lookup */ | ||
| type = 0; | ||
| } else { | ||
| type = _type_marker(value, state->_type_marker_str); |
Member
There was a problem hiding this comment.
We can further optimize the _type_marker function to use PyObject_GetOptionalAttr on >=3.13 like this:
static long _type_marker(PyObject* object, PyObject* _type_marker_str) {
PyObject* type_marker = NULL;
long type = 0;
#if PY_VERSION_HEX >= 0x030D0000
// 3.13
if (PyObject_GetOptionalAttr(object, _type_marker_str, &type_marker) == -1) {
return -1;
}
# else
if (PyObject_HasAttr(object, _type_marker_str)) {
type_marker = PyObject_GetAttr(object, _type_marker_str);
if (type_marker == NULL) {
return -1;
}
}
#endif
https://docs.python.org/3/c-api/object.html#c.PyObject_GetOptionalAttr
ShaneHarvey
previously approved these changes
Feb 26, 2026
NoahStapp
changed the title
NoahStapp
changed the title
sleepyStick
approved these changes
Feb 27, 2026
Contributor
sleepyStick
left a comment
sleepyStick
left a comment
There was a problem hiding this comment.
lol this sent me straight to reading docs land so it took me a hot minute to review but i've learned!
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PYTHON-5737
Changes in this PR
Add four optimizations to the C extension BSON implementation:
These changes result in the following BSON benchmark performance improvements on Python 3.14.3:
This represents up to a ~25% improvement for some benchmarks and brings them close to the high mark of Python 3.11 performance.
Test Plan
Verified with the existing test suite.
Checklist
Checklist for Author
Checklist for Reviewer