GH-46198: [Python] Remove deprecated PyExtensionType

[AlenkaF]

Rationale for this change

PyExtensionType has been deprecated in #38608 and can now be removed.

What changes are included in this PR?

PyExtensionType functionality is removed from PyArrow code.

Are these changes tested?

Existing tests should pass.

Are there any user-facing changes?

Deprecated classes are removed.

• GitHub Issue: [Python] Remove deprecated PyExtensionType #46198

[pitrou]

Can you check if it breaks https://pypi.org/project/pyarrow-hotfix/ ?

[AlenkaF]

Yeah, it breaks it:

>>> import pyarrow_hotfix
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/alenkafrim/Envs/pyarrow-dev/lib/python3.12/site-packages/pyarrow_hotfix/__init__.py", line 110, in <module>
    install()
  File "/Users/alenkafrim/Envs/pyarrow-dev/lib/python3.12/site-packages/pyarrow_hotfix/__init__.py", line 66, in install
    pa.unregister_extension_type("arrow.py_extension_type")
  File "pyarrow/types.pxi", line 2281, in pyarrow.lib.unregister_extension_type
    check_status(UnregisterPyExtensionType(c_type_name))
  File "pyarrow/error.pxi", line 92, in pyarrow.lib.check_status
    raise convert_status(status)
pyarrow.lib.ArrowKeyError: No type extension with name arrow.py_extension_type found

Should I propose an update in the pyarrow-hotfix repo __init__.py for PyArrow versions >= 21.0.0 similar to https://github.com/pitrou/pyarrow-hotfix/blob/main/src/pyarrow_hotfix/__init__.py#L39?

[pitrou]

@AlenkaF Yes, please do!

[AlenkaF]

@pitrou, I have submitted pitrou/pyarrow-hotfix#8 (tested locally with the changes in this PR and with pyarrow 19.0.0).

[AlenkaF]

As pyarrow-hotfix update has been merged, this PR is ready for review.
cc @raulcd @pitrou

[raulcd]

Can we remove it from the docs?

arrow/docs/source/python/api/datatypes.rst

Line 134 in 594fc16

PyExtensionType

[raulcd]

@github-actions crossbow submit -g python

[github-actions]

Revision: b4b3728

Submitted crossbow builds: ursacomputing/crossbow @ actions-2813be5290

Task	Status
example-python-minimal-build-fedora-conda
example-python-minimal-build-ubuntu-venv
test-conda-python-3.10
test-conda-python-3.10-hdfs-2.9.2
test-conda-python-3.10-hdfs-3.2.1
test-conda-python-3.10-pandas-latest-numpy-latest
test-conda-python-3.11
test-conda-python-3.11-dask-latest
test-conda-python-3.11-dask-upstream_devel
test-conda-python-3.11-hypothesis
test-conda-python-3.11-pandas-latest-numpy-1.26
test-conda-python-3.11-pandas-latest-numpy-latest
test-conda-python-3.11-pandas-nightly-numpy-nightly
test-conda-python-3.11-pandas-upstream_devel-numpy-nightly
test-conda-python-3.11-spark-master
test-conda-python-3.12
test-conda-python-3.12-cpython-debug
test-conda-python-3.13
test-conda-python-3.9
test-conda-python-3.9-pandas-1.1.3-numpy-1.19.5
test-conda-python-emscripten
test-cuda-python-ubuntu-22.04-cuda-11.7.1
test-debian-12-python-3-amd64
test-debian-12-python-3-i386
test-fedora-39-python-3
test-ubuntu-22.04-python-3
test-ubuntu-22.04-python-313-freethreading
test-ubuntu-24.04-python-3

[raulcd]

Thanks @AlenkaF

[conbench-apache-arrow]

After merging your PR, Conbench analyzed the 4 benchmarking runs that have been run so far on merge-commit 26170b4.

There were no benchmark performance regressions. 🎉

The full Conbench report has more details.