[DO NOT MERGE] increase number of blocks on spyre cards

[yannicks1]

changes:

• increasing number of blocks from 2080 to 8192

Note: do not merge yet until...

[github-actions]

👋 Hi! Thank you for contributing to vLLM support on Spyre.
Just a reminder: Make sure that your code passes all the linting checks, otherwise your PR won't be able to be merged. To do so, first install the linting requirements, then run format.sh and commit the changes. This can be done with uv directly:

uv sync --frozen --group lint --active --inexact

Or this can be done with pip:

uv pip compile --group lint > requirements-lint.txt
pip install -r requirements-lint.txt
bash format.sh

Now you are good to go 🚀

[joerunde]

@yannicks1 I posted this on our internal issue as well but we need to have some way to try to check the version of the spyre runtime stack so that we can set these values appropriately. We wouldn't want a newer version of vllm-spyre to set these expanded limits when its installed alongside an older version of the spyre runtime that doesn't support them.

(This would go away if we actually had APIs to call to get this data which we originally thought we would, but here we are 🤷 )

[yannicks1]

yeah, this makes sense! did not mean to merge this as is, but rather to have a branch to test this.
we can add the checks here once we know how to do it.

[yannicks1]

note: tkv x batchsize constraint will not be increased. number of blocks will be increased (for prefix caching)

[maxdebayser]

As travis mentioned, you could check the torch_sendnn version to see if it already has support 8K. The version can be found: torch_sendnn._version.__version__. But I think the code should use 8192 as the default and downgrade to 2080 if an old version of torch sendnn is found instead of the contrary because if someone is hacking on a local environment where they have editable installs directly from local git repos the version information might be wrong.