Add a much more detailed suite of tests for arb

[Maegereg]

Adapted from Tumult Core's tests for our own arb wrapper. I've modified these to work with the more bare-bones test setup python-flint uses since pytest or unittest don't seem to be idiomatic.

This does require adding a dev dependency on scipy - let me know if that's a problem. It makes the erf, erfc, and erfinv tests much easier to write, but it's not strictly necessary.

Tumult's arb wrapper supports hashing, so we have tests for it. I am inclined to leave them in and add hashing in a future PR, but let me know if that's not a feature we want.

[oscarbenjamin]

This has caused the tests to fail in CI but I think that the math module's versions of these functions should be fine.

[Maegereg]

Done.

[oscarbenjamin]

On hashing currently an arb does not necessarily compare equal to itself:

In [1]: from flint import arb

In [2]: a = arb(1, 0.0001)

In [3]: a
Out[3]: [1.000 +/- 1.01e-4]

In [4]: a == a
Out[4]: False

I'm not sure about that behaviour but it does make hashing less useful.

[Maegereg]

Looks like arb doesn't have an __eq__ defined. I'm happy to add one of those at the same time.

[oscarbenjamin]

Arb uses __richcmp__ so it does define == but it uses arb_eq which is defined as returning False unless both arbs are exact and exactly equal.

[oscarbenjamin]

There should be a test that all the tests in this file are in this list or otherwise someone will end up adding tests here without adding them to the list. There is a test function that does this in test_all.py which could be adapted to do check for this file as well.

[Maegereg]

Ah, I was wondering how that was handled. Done.

Have y'all considered switching to a test framework like pytest rather than implementing all of this manually?

[oscarbenjamin]

You can use pytest as well. The idea is just that you can run

python -m flint.test

to test your installation without needing to install anything.

Usually for development I would run spin test which ultimately calls pytest --pyargs flint.

[Maegereg]

Fair enough! I do think requiring pytest would make the process of writing tests easier (and given that you're aiming to add more tests that might be a worthwhile tradeoff), but I can see why the ability to check the installation easily is useful too.

[oscarbenjamin]

We could require pytest but I don't think it would particularly make writing tests easier. Either way you mostly just write a lot of functions called test_foo. The potential advantages of pytest are things like selecting particular tests or running tests in parallel but the test suite is fast enough not to need those. One thing pytest is not good at is doctests which make up a good fraction of the test suite.

[Maegereg]

Sounds good. I think this should be in good shape - do you want me to resolve threads, or should I leave it to you?

[Maegereg]

Arb uses richcmp so it does define == but it uses arb_eq which is defined as returning False unless both arbs are exact and exactly equal.

Ah, got it, that all makes sense now. In an ideal world you'd really want to use arb_equal for dictionary comparisons rather than arb_eq, but we only get one __eq__ method. Still, I think hashing is situationally useful (we use it for a lru_cache where the arbs will all be exact), so I plan to add it unless someone objects.

[oscarbenjamin]

Now that tests are passed I think this looks good so let's get it in. Thanks!

[oscarbenjamin]

Still, I think hashing is situationally useful (we use it for a lru_cache where the arbs will all be exact),

Arbs are immutable so hashing makes sense. It obviously needs to match equality though so as it stands it would only be useful for exact arbs.