diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 250297d7a85b..eae13edb6c10 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -24,7 +24,7 @@ it takes a bit longer. For more details, read below. ## Preparing the environment -### Code away! +### Code away Typeshed runs continuous integration (CI) on all pull requests. This means that if you file a pull request (PR), our full test suite @@ -36,6 +36,16 @@ This frees you up to ignore all local setup on your side, focus on the code and rely on the CI to fix everything, or point you to the places that need fixing. +### Code formatting + +Typeshed uses `Black`, `Ruff`, and `Flake8` (with `flake8-pyi`) to enforce +formatting and style. CI will automatically apply many formatting fixes for +you, but you can run the same checks locally via `pre-commit`: + +```bash +(.venv)$ pre-commit run --all-files +``` + ### ... Or create a local development environment If you prefer to run the tests and formatting locally, it's @@ -50,56 +60,39 @@ please refer to this Note that some tests require extra setup steps to install the required dependencies. - - - - - - - - - - - - - - - -
Linux / macOS / WSL - - On Linux and macOS, you will be able to run the full test suite on Python - 3.9-3.12. - To install the necessary requirements, run the following commands from a - terminal window: - - ```bash - $ python3 -m venv .venv - $ source .venv/bin/activate - (.venv)$ pip install -U pip - (.venv)$ pip install -r requirements-tests.txt - ``` - -
Windows - - Run the following commands from a Windows terminal to install all requirements: - - ```powershell - > py -m venv .venv - > .venv\Scripts\activate - (.venv) > python -m pip install -U pip - (.venv) > pip install -r requirements-tests.txt - ``` - -
Using uv - - If you already have [uv](https://docs.astral.sh/uv/getting-started/installation/) installed, you can simply replace the commands above with: - - ```shell - uv venv - uv pip install -r requirements-tests.txt - ``` - -
+### Linux / macOS / WSL + +On Linux and macOS, you will be able to run the full test suite on Python +3.9-3.12. +To install the necessary requirements, run the following commands from a +terminal window: + +```bash +$ python3 -m venv .venv +$ source .venv/bin/activate +(.venv)$ pip install -U pip +(.venv)$ pip install -r requirements-tests.txt +``` + +### Windows + +Run the following commands from a Windows terminal to install all requirements: + +```powershell +> py -m venv .venv +> .venv\Scripts\activate +(.venv) > python -m pip install -U pip +(.venv) > pip install -r requirements-tests.txt +``` + +### Using uv + +If you already have [uv](https://docs.astral.sh/uv/getting-started/installation/) installed, you can simply replace the commands above with: + +```shell +uv venv +uv pip install -r requirements-tests.txt +``` ## Where to make changes @@ -121,6 +114,7 @@ to use syntax and typing features not supported by that version. ### Third-party library stubs We accept stubs for third-party packages into typeshed as long as: + * the package is publicly available on the [Python Package Index](https://pypi.org/); * the package supports any Python version supported by typeshed; and * the package does not ship with its own stubs or type annotations. @@ -129,6 +123,7 @@ The fastest way to generate new stubs is to use `scripts/create_baseline_stubs.p Stubs for third-party packages go into the `stubs` directory. Each subdirectory there represents a PyPI distribution, and contains the following: + * `METADATA.toml`, describing the package. See below for details. * Stubs (i.e. `*.pyi` files) for packages and modules that are shipped in the source distribution. @@ -155,9 +150,9 @@ supported: * `version`: The versions of the library that the stubs support. Two formats are supported: - - A concrete version. This is especially suited for libraries that + * A concrete version. This is especially suited for libraries that use [Calendar Versioning](https://calver.org/). - - A version range ending in `.*`. This is suited for libraries that + * A version range ending in `.*`. This is suited for libraries that reflect API changes in the version number only, where the API-independent part is represented by the asterisk. In the case of [Semantic Versioning](https://semver.org/), this version could look @@ -197,6 +192,7 @@ supported: In addition, we specify configuration for stubtest in the `tool.stubtest` table. This has the following keys: + * `skip` (default: `false`): Whether stubtest should be run against this package. Please avoid setting this to `true`, and add a comment if you have to. @@ -235,7 +231,6 @@ distribution. The format of all `METADATA.toml` files can be checked by running `python3 ./tests/check_typeshed_structure.py`. - ## Making Changes ### Before you begin @@ -279,8 +274,9 @@ constants, etc.) of the module they cover, but it is not always clear exactly what is part of the interface. The following should always be included: -- All objects listed in the module's documentation. -- All objects included in ``__all__`` (if present). + +* All objects listed in the module's documentation. +* All objects included in ``__all__`` (if present). Other objects may be included if they are being used in practice or if they are not prefixed with an underscore. This means @@ -350,35 +346,36 @@ be used in typeshed as soon as the PEP has been accepted and implemented and most type checkers support the new feature. Supported features include: -- [PEP 544](https://peps.python.org/pep-0544/) (Protocol) -- [PEP 585](https://peps.python.org/pep-0585/) (builtin generics) -- [PEP 586](https://peps.python.org/pep-0586/) (Literal) -- [PEP 591](https://peps.python.org/pep-0591/) (Final/@final) -- [PEP 589](https://peps.python.org/pep-0589/) (TypedDict) -- [PEP 604](https://peps.python.org/pep-0604/) (`Foo | Bar` union syntax) -- [PEP 612](https://peps.python.org/pep-0612/) (ParamSpec) -- [PEP 647](https://peps.python.org/pep-0647/) (TypeGuard): + +* [PEP 544](https://peps.python.org/pep-0544/) (Protocol) +* [PEP 585](https://peps.python.org/pep-0585/) (builtin generics) +* [PEP 586](https://peps.python.org/pep-0586/) (Literal) +* [PEP 591](https://peps.python.org/pep-0591/) (Final/@final) +* [PEP 589](https://peps.python.org/pep-0589/) (TypedDict) +* [PEP 604](https://peps.python.org/pep-0604/) (`Foo | Bar` union syntax) +* [PEP 612](https://peps.python.org/pep-0612/) (ParamSpec) +* [PEP 647](https://peps.python.org/pep-0647/) (TypeGuard): see [#5406](https://github.com/python/typeshed/issues/5406) -- [PEP 655](https://peps.python.org/pep-0655/) (`Required` and `NotRequired`) -- [PEP 673](https://peps.python.org/pep-0673/) (`Self`) -- [PEP 675](https://peps.python.org/pep-0675/) (`LiteralString`) -- [PEP 702](https://peps.python.org/pep-0702/) (`@deprecated()`) +* [PEP 655](https://peps.python.org/pep-0655/) (`Required` and `NotRequired`) +* [PEP 673](https://peps.python.org/pep-0673/) (`Self`) +* [PEP 675](https://peps.python.org/pep-0675/) (`LiteralString`) +* [PEP 702](https://peps.python.org/pep-0702/) (`@deprecated()`) Features from the `typing` module that are not present in all supported Python versions must be imported from `typing_extensions` instead in typeshed stubs. This currently affects: -- `TypeAlias` (new in Python 3.10) -- `Concatenate` (new in Python 3.10) -- `ParamSpec` (new in Python 3.10) -- `TypeGuard` (new in Python 3.10) -- `Self` (new in Python 3.11) -- `Never` (new in Python 3.11) -- `LiteralString` (new in Python 3.11) -- `TypeVarTuple` and `Unpack` (new in Python 3.11) -- `Required` and `NotRequired` (new in Python 3.11) -- `Buffer` (new in Python 3.12; in the `collections.abc` module) -- `@deprecated` (new in Python 3.13; in the `warnings` module) +* `TypeAlias` (new in Python 3.10) +* `Concatenate` (new in Python 3.10) +* `ParamSpec` (new in Python 3.10) +* `TypeGuard` (new in Python 3.10) +* `Self` (new in Python 3.11) +* `Never` (new in Python 3.11) +* `LiteralString` (new in Python 3.11) +* `TypeVarTuple` and `Unpack` (new in Python 3.11) +* `Required` and `NotRequired` (new in Python 3.11) +* `Buffer` (new in Python 3.12; in the `collections.abc` module) +* `@deprecated` (new in Python 3.13; in the `warnings` module) Some type checkers implicitly promote the `bytearray` and `memoryview` types to `bytes`. @@ -462,10 +459,10 @@ We sometimes use the ["status: deferred" label](https://github.com/python/typesh to mark PRs and issues that we'd like to accept, but that are blocked by some external factor. Blockers can include: -- An unambiguous bug in a type checker (i.e., a case where the +* An unambiguous bug in a type checker (i.e., a case where the type checker is not implementing [the typing spec](https://typing.readthedocs.io/en/latest/spec/index.html)). -- A dependency on a typing PEP that is still under consideration. -- A pending change in a related project, such as stub-uploader. +* A dependency on a typing PEP that is still under consideration. +* A pending change in a related project, such as stub-uploader. ### Closing stale PRs diff --git a/MAINTAINERS.md b/MAINTAINERS.md index 7f33b516f22b..f85a71b899a6 100644 --- a/MAINTAINERS.md +++ b/MAINTAINERS.md @@ -1,4 +1,4 @@ -At present the active maintainers are (alphabetically): +# At present the active maintainers are (alphabetically) * Rebecca Chen (@rchen152) * Jukka Lehtosalo (@JukkaL) diff --git a/README.md b/README.md index d295b56bc05d..471ed31454cf 100644 --- a/README.md +++ b/README.md @@ -19,7 +19,7 @@ it before submitting pull requests; do not report issues with annotations to the project the stubs are for, but instead report them here to typeshed.** Further documentation on stub files, typeshed, and Python's typing system in -general, can also be found at https://typing.readthedocs.io/en/latest/. +general, can also be found at . Typeshed supports Python versions 3.9 to 3.14. @@ -35,7 +35,7 @@ be installed from PyPI. For example, if you are using `html5lib` and `requests`, you can install the type stubs using ```bash -$ pip install types-html5lib types-requests +pip install types-html5lib types-requests ``` These PyPI packages follow [PEP 561](http://www.python.org/dev/peps/pep-0561/) diff --git a/requirements-tests.txt b/requirements-tests.txt index 9a6db80b26c1..db44ea667728 100644 --- a/requirements-tests.txt +++ b/requirements-tests.txt @@ -1,6 +1,6 @@ # Type checkers that we test our stubs against. These should always # be pinned to a specific version to make failure reproducible. -mypy==1.18.2 +mypy==1.19.1 pyright==1.1.407 # Libraries used by our various scripts. diff --git a/stubs/Send2Trash/METADATA.toml b/stubs/Send2Trash/METADATA.toml index 8ff27a7205d0..ef3d502bb9f6 100644 --- a/stubs/Send2Trash/METADATA.toml +++ b/stubs/Send2Trash/METADATA.toml @@ -1,2 +1,2 @@ -version = "1.8.*" +version = "2.0.*" upstream_repository = "https://github.com/arsenetar/send2trash" diff --git a/stubs/aiofiles/aiofiles/threadpool/text.pyi b/stubs/aiofiles/aiofiles/threadpool/text.pyi index 6c6bf20a408c..c2847e4174ee 100644 --- a/stubs/aiofiles/aiofiles/threadpool/text.pyi +++ b/stubs/aiofiles/aiofiles/threadpool/text.pyi @@ -1,4 +1,3 @@ -from _typeshed import FileDescriptorOrPath from collections.abc import Iterable from typing import BinaryIO, type_check_only @@ -35,7 +34,7 @@ class _UnknownAsyncTextIO(AsyncBase[str]): @property def newlines(self) -> str | tuple[str, ...] | None: ... @property - def name(self) -> FileDescriptorOrPath: ... + def name(self) -> str: ... @property def mode(self) -> str: ... diff --git a/stubs/auth0-python/METADATA.toml b/stubs/auth0-python/METADATA.toml index 4e1606df7422..a85c61332be8 100644 --- a/stubs/auth0-python/METADATA.toml +++ b/stubs/auth0-python/METADATA.toml @@ -1,3 +1,3 @@ -version = "4.10.*" +version = "4.13.*" upstream_repository = "https://github.com/auth0/auth0-python" requires = ["cryptography", "types-requests"] diff --git a/stubs/cffi/METADATA.toml b/stubs/cffi/METADATA.toml index abf41186a33b..22f51de9ed5b 100644 --- a/stubs/cffi/METADATA.toml +++ b/stubs/cffi/METADATA.toml @@ -1,4 +1,4 @@ -version = "1.17.*" +version = "2.0.*" upstream_repository = "https://github.com/python-cffi/cffi/" requires = ["types-setuptools"] diff --git a/stubs/docutils/METADATA.toml b/stubs/docutils/METADATA.toml index 1a84d3665e0c..6e85afb83c18 100644 --- a/stubs/docutils/METADATA.toml +++ b/stubs/docutils/METADATA.toml @@ -1,2 +1,2 @@ -version = "0.22.3" +version = "0.22.4" upstream_repository = "https://sourceforge.net/p/docutils/code" diff --git a/stubs/fpdf2/METADATA.toml b/stubs/fpdf2/METADATA.toml index 99ca1632cb80..798fcf1d018c 100644 --- a/stubs/fpdf2/METADATA.toml +++ b/stubs/fpdf2/METADATA.toml @@ -1,4 +1,4 @@ -version = "2.8.4" +version = "2.8.5" upstream_repository = "https://github.com/PyFPDF/fpdf2" requires = ["Pillow>=10.3.0"] diff --git a/stubs/hdbcli/METADATA.toml b/stubs/hdbcli/METADATA.toml index 168d2786a79a..c7d8c8e65386 100644 --- a/stubs/hdbcli/METADATA.toml +++ b/stubs/hdbcli/METADATA.toml @@ -1,2 +1,2 @@ -version = "2.25.*" +version = "2.27.*" # upstream_repository = closed-source diff --git a/stubs/ibm-db/METADATA.toml b/stubs/ibm-db/METADATA.toml index 6f7b307e221e..e754728c4eb2 100644 --- a/stubs/ibm-db/METADATA.toml +++ b/stubs/ibm-db/METADATA.toml @@ -1,2 +1,2 @@ -version = "3.2.6" +version = "3.2.8" upstream_repository = "https://github.com/ibmdb/python-ibmdb" diff --git a/stubs/protobuf/METADATA.toml b/stubs/protobuf/METADATA.toml index fc8dfe341705..a9af8bcb5317 100644 --- a/stubs/protobuf/METADATA.toml +++ b/stubs/protobuf/METADATA.toml @@ -1,6 +1,6 @@ # Using an exact number in the specifier for scripts/sync_protobuf/google_protobuf.py # When updating, also re-run the script -version = "~=6.32.1" +version = "~=6.33.2" upstream_repository = "https://github.com/protocolbuffers/protobuf" extra_description = "Partially generated using [mypy-protobuf==3.6.0](https://github.com/nipunn1313/mypy-protobuf/tree/v3.6.0) and libprotoc 31.1 on [protobuf v32.1](https://github.com/protocolbuffers/protobuf/releases/tag/v32.1) (python `protobuf==6.32.1`)." partial_stub = true diff --git a/stubs/pyperclip/METADATA.toml b/stubs/pyperclip/METADATA.toml index 885e7747f335..9e2c211f627d 100644 --- a/stubs/pyperclip/METADATA.toml +++ b/stubs/pyperclip/METADATA.toml @@ -1,4 +1,4 @@ -version = "1.9.*" +version = "1.11.*" upstream_repository = "https://github.com/asweigart/pyperclip" [tool.stubtest] diff --git a/stubs/reportlab/reportlab/lib/rltempfile.pyi b/stubs/reportlab/reportlab/lib/rltempfile.pyi index 24b270ff237c..2cb9bb42992c 100644 --- a/stubs/reportlab/reportlab/lib/rltempfile.pyi +++ b/stubs/reportlab/reportlab/lib/rltempfile.pyi @@ -1,4 +1,4 @@ def get_rl_tempdir(*subdirs: str) -> str: ... def get_rl_tempfile(fn: str | None = None) -> str: ... -__all__ = ("get_rl_tempdir", "get_rl_tempdir") +__all__ = ("get_rl_tempdir", "get_rl_tempfile") diff --git a/stubs/tensorflow/METADATA.toml b/stubs/tensorflow/METADATA.toml index a207ca05601a..027aab52a824 100644 --- a/stubs/tensorflow/METADATA.toml +++ b/stubs/tensorflow/METADATA.toml @@ -1,6 +1,6 @@ # Using an exact number in the specifier for scripts/sync_protobuf/tensorflow.py # When updating, also re-run the script -version = "~=2.18.0" +version = "~=2.20.0" upstream_repository = "https://github.com/tensorflow/tensorflow" # requires a version of numpy with a `py.typed` file requires = ["numpy>=1.20", "types-protobuf", "types-requests"] diff --git a/tests/README.md b/tests/README.md index a8f2c5e9a944..1aeb9fe57a9c 100644 --- a/tests/README.md +++ b/tests/README.md @@ -1,4 +1,5 @@ -This directory contains several tests: +# This directory contains several tests + - `tests/mypy_test.py` tests the stubs with [mypy](https://github.com/python/mypy/) - `tests/pyright_test.py` tests the stubs with @@ -21,6 +22,7 @@ In order for `pyright_test` to work correctly, some third-party stubs may require extra dependencies external to typeshed to be installed in your virtual environment prior to running the test. You can list or install all of a stubs package's external dependencies using the following script: + ```bash (.venv3)$ python tests/get_external_stub_requirements.py # List external dependencies for (.venv3)$ python tests/get_external_stub_requirements.py # List external dependencies for and @@ -31,6 +33,7 @@ You can list or install all of a stubs package's external dependencies using the ## Run all tests for a specific stub Run using: + ```bash (.venv3)$ python3 tests/runtests.py / ``` @@ -51,6 +54,7 @@ For more information, see the docs on [`stubtest_stdlib.py`](#stubtest_stdlibpy) ## mypy\_test.py Run using: + ```bash (.venv3)$ python3 tests/mypy_test.py ``` @@ -70,6 +74,7 @@ for this script. This test requires [Node.js](https://nodejs.org) to be installed. Although typeshed runs pyright in CI, it does not currently use this script. However, this script uses the same pyright version and configuration as the CI. + ```bash (.venv3)$ python3 tests/pyright_test.py # Check all files (.venv3)$ python3 tests/pyright_test.py stdlib/sys.pyi # Check one file @@ -95,13 +100,15 @@ for information on the various configuration options. This checks that typeshed's directory structure and metadata files are correct. Run using: + ```bash -$ python3 tests/check_typeshed_structure.py +python3 tests/check_typeshed_structure.py ``` ## stubtest\_stdlib.py Run using + ```bash (.venv3)$ python3 tests/stubtest_stdlib.py ``` @@ -137,6 +144,7 @@ this script locally if you know you can trust the packages you're running stubtest on. Run using + ```bash (.venv3)$ python3 tests/stubtest_third_party.py ``` @@ -151,6 +159,7 @@ check on the command line: If you have the runtime package installed in your local virtual environment, you can also run stubtest directly, with + ```bash (.venv3)$ MYPYPATH= python3 -m mypy.stubtest \ --custom-typeshed-dir \ @@ -202,6 +211,7 @@ configuration to properly validate Django-specific types during stubtest executi ## typecheck\_typeshed.py Run using + ```bash (.venv3)$ python3 tests/typecheck_typeshed.py ``` diff --git a/tests/REGRESSION.md b/tests/REGRESSION.md index e0d37edd56f1..9eb6c3d3544b 100644 --- a/tests/REGRESSION.md +++ b/tests/REGRESSION.md @@ -1,4 +1,4 @@ -## Regression tests for typeshed +# Regression tests for typeshed Regression tests for the standard library stubs can be found in the `stdlib/@tests/test_cases` directory. Not all third-party-library stub @@ -13,7 +13,7 @@ get right.** 100% test coverage for typeshed is neither necessary nor desirable, as it would lead to code duplication. Moreover, typeshed has multiple other mechanisms for spotting errors in the stubs. -### The purpose of these tests +## The purpose of these tests Different test cases in typeshed serve different purposes. For some typeshed stubs, the type annotations are complex enough that it's useful to have @@ -126,7 +126,7 @@ test cases for `foo` should be put in a file named This means that mypy will only run the test case if `--python-version 3.11`, `--python-version 3.12`, etc. is passed on the command line to `tests/regr_test.py`, -but it _won't_ run the test case if e.g. `--python-version 3.9` +but it *won't* run the test case if e.g. `--python-version 3.9` is passed on the command line. However, `if sys.version_info >= (3, target):` is still required for `pyright` diff --git a/tests/stubtest_third_party.py b/tests/stubtest_third_party.py index d2ea4190bc77..98e553ebb3b0 100755 --- a/tests/stubtest_third_party.py +++ b/tests/stubtest_third_party.py @@ -236,6 +236,11 @@ def setup_gdb_stubtest_command(venv_dir: Path, stubtest_cmd: list[str]) -> bool: # Taken from https://github.com/pwndbg/pwndbg/blob/83d8d95b576b749e888f533ce927ad5a77fb957b/gdbinit.py#L37 site_pkgs_path = glob(os.path.join({str(venv_dir)!r}, "lib/*/site-packages"))[0] site.addsitedir(site_pkgs_path) + print("!!! adding site", site_pkgs_path) + print("!!! sys.path", sys.path) + + import subprocess + subprocess.run("find / | grep librt", shell=True) exit_code = 1 try: