Skip to content

feat: lazy imports google auth#17679

Open
hebaalazzeh wants to merge 20 commits into
mainfrom
feature/lazy-imports-google-auth
Open

feat: lazy imports google auth#17679
hebaalazzeh wants to merge 20 commits into
mainfrom
feature/lazy-imports-google-auth

Conversation

@hebaalazzeh

@hebaalazzeh hebaalazzeh commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Description

This PR initiates the rollout of our PEP 0810 lazy-loading architecture to google-auth, starting with the transport module.

By applying Python 3.15's native explicit lazy imports directly inside our transport wrappers (requests.py, urllib3.py, and grpc.py), we defer the eager parsing and compilation of the heavy third-party networking libraries (requests, urllib3, and grpc). This acts as the first step in addressing the high initialization latency and peak memory footprints we're seeing on Serverless cold-starts.

Older Python runtimes (3.14 and below) safely ignore the __lazy_modules__ set and fall back to standard eager execution, meaning this introduces zero backwards compatibility risk.

Related Documents

Note: This PR was kept intentionally small for review speed. We will be executing an iterative rollout, extending this pattern to other heavy modules like oauth2 and compute_engine in fast follow-up PRs once this structural pattern is approved.

@hebaalazzeh hebaalazzeh marked this pull request as ready for review July 9, 2026 05:09
@hebaalazzeh hebaalazzeh requested review from a team as code owners July 9, 2026 05:09

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a __lazy_modules__ set in packages/google-auth/google/auth/transport/__init__.py to support lazy loading of transport modules. The review feedback correctly identifies an incorrect module name (_aiohttp_requests instead of aiohttp_requests) that would prevent lazy importing in Python 3.15+.

Comment thread packages/google-auth/google/auth/transport/__init__.py Outdated

@parthea parthea left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM but holding off on formal approval until we have tests. We should assert that on Python 3.15+ the modules are indeed absent from sys.modules until accessed, and that on pre-3.15 environments they fallback cleanly to standard eager imports

import sys
import pytest

# List of modules we expect to be lazy
LAZY_MODULES = [
    "google.auth.transport.requests",
    "google.auth.transport.urllib3",
    "google.auth.transport.grpc",
]

def clean_sys_modules():
    """Helper to ensure we start with a clean slate for import testing."""
    for mod in LAZY_MODULES:
        sys.modules.pop(mod, None)

@pytest.mark.skipif(sys.version_info < (3, 15), reason="PEP 810 requires Python 3.15+")
def test_lazy_imports_on_python_315():
    clean_sys_modules()
    
    # 1. Import the transport package
    import google.auth.transport
    
    # 2. Assert that none of the lazy modules have been eagerly loaded into sys.modules
    for mod in LAZY_MODULES:
        assert mod not in sys.modules
        
    # 3. Access an attribute to trigger reification
    from google.auth.transport import requests
    _ = requests.__name__  # Trigger first-use reification
    
    # 4. Assert that the module has now been reified and loaded
    assert "google.auth.transport.requests" in sys.modules


@pytest.mark.skipif(sys.version_info >= (3, 15), reason="Testing fallback behavior on < 3.15")
def test_fallback_eager_imports_pre_315():
    clean_sys_modules()
    
    # On older Python, __lazy_modules__ is safely ignored, meaning they should eager-load
    import google.auth.transport
    
    for mod in LAZY_MODULES:
        assert mod in sys.modules

@parthea

parthea commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

I see the tests are failing. We also need to add from google.auth.transport import requests to the test file

@parthea

parthea commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Disregard my last comment.

Under PEP 810, __lazy_modules__ is a module-local opt-in . It only intercepts import statements written inside the module where it is defined

We should only add __lazy_modules__ to files where we also have the same import statements

From https://docs.python.org/3.15/reference/simple_stmts.html#lazy-imports

Any regular (non-lazy) import statement at module scope whose target appears in lazy_modules is treated as a lazy import, exactly as if the lazy keyword had been used.

Comment on lines +27 to +43
# PEP 0810: Explicit Lazy Imports
# Python 3.15+ natively intercepts and defers these imports.
# Developers can disable this behavior and force eager imports.
# For more information, see:
# https://docs.python.org/3.15/library/sys.html#sys.set_lazy_imports_filter
# Older Python versions safely ignore this variable.
__lazy_modules__ = {
"google.auth.transport.aiohttp_requests",
"google.auth.transport._custom_tls_signer",
"google.auth.transport._http_client",
"google.auth.transport._mtls_helper",
"google.auth.transport._requests_base",
"google.auth.transport.grpc",
"google.auth.transport.mtls",
"google.auth.transport.requests",
"google.auth.transport.urllib3",
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove this code since we don't have the corresponding import statements in this file

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

addressed

@hebaalazzeh hebaalazzeh requested a review from a team as a code owner July 10, 2026 01:33
@hebaalazzeh hebaalazzeh force-pushed the feature/lazy-imports-google-auth branch from 6e6fe6e to eef11dc Compare July 10, 2026 06:43
@hebaalazzeh

Copy link
Copy Markdown
Contributor Author

addressed all comments

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants