`snyk test` is broken in virtual environments without `pip` e.g. created by `uv`

[jmahlik]

In a virtual environment where pip is not importable, the snyk python plugin fails due to a reliance on the deprecated pkg_resources module to determine the working set.

As new tools like uv become more prevalent. More environments will no longer have pip installed by default. We encountered this when running snyk from venvs created by uv. As far as I can tell, this should be supported because the environments follow the virtual environment PEP 405.

Reproducer

uv venv venv
. venv/bin/activate
uv pip install blurb
uv pip freeze > requirements.txt
snyk test --file=requirements.txt --org=$SNYK_ORG --json-file-output=snyk.json --severity-threshold=high --debug --log-level=trace

2025-02-06T19:41:19.060Z snyk:run-test Error running test {
  error: 'Traceback (most recent call last):\n' +
    '  File "/tmp/tmp-357-IyPnbx3NFTYb/pip_resolve.py", line 7, in <module>\n' +
    '    import utils\n' +
    '  File "/tmp/tmp-357-IyPnbx3NFTYb/utils.py", line 3, in <module>\n' +
    '    import pip_resolve\n' +
    '  File "/tmp/tmp-357-IyPnbx3NFTYb/pip_resolve.py", line 8, in <module>\n' +
    '    import requirements\n' +
    '  File "/tmp/tmp-357-IyPnbx3NFTYb/requirements/__init__.py", line 1, in <module>\n' +
    '    from .parser import parse   # noqa\n' +
    '    ^^^^^^^^^^^^^^^^^^^^^^^^^\n' +
    '  File "/tmp/tmp-357-IyPnbx3NFTYb/requirements/parser.py", line 7, in <module>\n' +
    '    from .requirement import Requirement\n' +
    '  File "/tmp/tmp-357-IyPnbx3NFTYb/requirements/requirement.py", line 3, in <module>\n' +
    '    from pkg_resources import Requirement as Req\n' +
    '  File "/tmp/tmp-357-IyPnbx3NFTYb/pkg_resources.py", line 5, in <module>\n' +
    '    from pkg_resources_py3_12 import *\n' +
    '  File "/tmp/tmp-357-IyPnbx3NFTYb/pkg_resources_py3_12/__init__.py", line 79, in <module>\n' +
    '    from pip._internal.utils._jaraco_text import (\n' +
    "ModuleNotFoundError: No module named 'pip'\n"
}

Possible solution

Recommend moving away from pkg_resources. Using packaging or importlib.metadata.distributions() over pkg_resources is a good option.

Here's a quick way to get all the installed packages using importlib metadata.

importlib.metadata
excluded = {
    "pip",
    "setuptools",
}
packages = [
    f"{dist.metadata['Name']}=={dist.version}"
    for dist in importlib.metadata.distributions()
    if dist.metadata["Name"] not in excluded
]

Related to this issue #226 where setuptools was assumed to be in the environment.

"This may be a good time to move away from pkg_resources to get the working set as well. It has been deprecated for a long time and has some nasty side effects on import. It may stop functioning one day. I realize it's vendored, but might be good to get away from it if possible."

It seems to be that time now. Importing private pip apis is one of the nasty side effects.

[jmahlik]

I should note the workaround is installing pip into the environment. But it's non-obvious and fixable.