Skip to content

Conversation

@AlenkaF
Copy link
Contributor

@AlenkaF AlenkaF commented Apr 23, 2025

In PyArrow version 21.0.0 PyExtensionType will be removed, see apache/arrow#46199. We need to update the checks in __init__ so that the hotfix doesn't break with future pyarrow versions.

@pitrou
Copy link
Owner

pitrou commented Apr 23, 2025

Hi @AlenkaF , two things:

  1. can you merge from main to fix existing CI tests (I had to update them to account for GitHub Actions runner updates)
  2. can you check the hotfix tests run with PyArrow 21?

@AlenkaF AlenkaF force-pushed the update-for-pyarrow-21 branch from 89315f9 to 28b8b25 Compare April 23, 2025 12:22
@AlenkaF
Copy link
Contributor Author

AlenkaF commented Apr 23, 2025

can you merge from main to fix existing CI tests (I had to update them to account for GitHub Actions runner updates)

Done.

can you check the hotfix tests run with PyArrow 21?

With the last commit and hacky diff for local dev setup:

diff --git a/tests/test_in_process.py b/tests/test_in_process.py
index 19bf894..0f87759 100644
--- a/tests/test_in_process.py
+++ b/tests/test_in_process.py
@@ -9,7 +9,7 @@ import pyarrow.parquet as pq
 import pyarrow_hotfix
 
 
-pa_version = tuple(map(int, pa.__version__.split('.')))
+pa_version = tuple(map(int, ("21.0.0").split('.')))

the tests passed.

> pytest tests
========================================================================== test session starts ==========================================================================
platform darwin -- Python 3.12.10, pytest-8.3.5, pluggy-1.5.0
rootdir: /Users/alenkafrim/Repos/pyarrow-hotfix
configfile: pyproject.toml
plugins: anyio-4.9.0, hypothesis-6.127.4
collected 14 items                                                                                                                                                      

tests/test_in_process.py .......                                                                                                                                  [ 50%]
tests/test_subprocess.py .......                                                                                                                                  [100%]

========================================================================== 14 passed in 2.00s ===========================================================================

Adding package info, just in case:

>>> pip list
...
pyarrow                       20.0.0.dev327+gb4b3728ff4 /Users/alenkafrim/Repos/arrow/python
pyarrow-hotfix                0.6                       /Users/alenkafrim/Repos/pyarrow-hotfix

@AlenkaF AlenkaF marked this pull request as ready for review April 23, 2025 12:28
@pitrou
Copy link
Owner

pitrou commented Apr 23, 2025

It seems this needs to be fixed for PyArrow 0.14, see CI failures.

@AlenkaF
Copy link
Contributor Author

AlenkaF commented Apr 23, 2025

Plan to have a look at it tomorrow/Friday. Thanks!

@AlenkaF
Copy link
Contributor Author

AlenkaF commented Apr 24, 2025

@pitrou pinging for CI approval.

Copy link
Collaborator

@raulcd raulcd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @AlenkaF !

@pitrou pitrou merged commit 8d44d72 into pitrou:main Apr 25, 2025
36 checks passed
@pitrou
Copy link
Owner

pitrou commented Apr 25, 2025

I'm gonna make a new release soon.

@pitrou
Copy link
Owner

pitrou commented Apr 25, 2025

The release is now made: https://pypi.org/project/pyarrow-hotfix/

@AlenkaF AlenkaF deleted the update-for-pyarrow-21 branch April 25, 2025 12:48
raulchen added a commit to ray-project/ray that referenced this pull request May 6, 2025
Upgrade `pyarrow-hotfix` to fix compatibility issue with the latest
pyarrow nightly. See pitrou/pyarrow-hotfix#8

Signed-off-by: Hao Chen <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants