add EESSIError exception to avoid printing traceback by smoors · Pull Request #319 · EESSI/test-suite

smoors · 2026-01-31T15:05:48Z

changes this:

ERROR: run session stopped: reframe fatal error: Invalid thread_binding value 'nonexisting'. Valid values: 'true', 'compact', or 'false'.
ERROR: Traceback (most recent call last):
  File "/vscmnt/brussel_pixiu_data/_data_brussel/100/vsc10009/reframe/eessitestsuite/venv-reframe/lib64/python3.9/site-packages/reframe/frontend/cli.py", line 1354, in main
    checks_found = loader.load_all(force=True)
  File "/vscmnt/brussel_pixiu_data/_data_brussel/100/vsc10009/reframe/eessitestsuite/venv-reframe/lib64/python3.9/site-packages/reframe/core/logging.py", line 1121, in _fn
    return fn(*args, **kwargs)
  File "/vscmnt/brussel_pixiu_data/_data_brussel/100/vsc10009/reframe/eessitestsuite/venv-reframe/lib64/python3.9/site-packages/reframe/frontend/loader.py", line 244, in load_all
    checks += self.load_from_dir(d, self._recurse, force)
  File "/vscmnt/brussel_pixiu_data/_data_brussel/100/vsc10009/reframe/eessitestsuite/venv-reframe/lib64/python3.9/site-packages/reframe/frontend/loader.py", line 223, in load_from_dir
    checks += self.load_from_file(entry.path, force)
  File "/vscmnt/brussel_pixiu_data/_data_brussel/100/vsc10009/reframe/eessitestsuite/venv-reframe/lib64/python3.9/site-packages/reframe/frontend/loader.py", line 195, in load_from_file
    return self.load_from_module(util.import_module_from_file(
  File "/vscmnt/brussel_pixiu_data/_data_brussel/100/vsc10009/reframe/eessitestsuite/venv-reframe/lib64/python3.9/site-packages/reframe/frontend/loader.py", line 156, in load_from_module
    candidate_tests = registry.instantiate_all(reset_sysenv,
  File "/vscmnt/brussel_pixiu_data/_data_brussel/100/vsc10009/reframe/eessitestsuite/venv-reframe/lib64/python3.9/site-packages/reframe/core/logging.py", line 1121, in _fn
    return fn(*args, **kwargs)
  File "/vscmnt/brussel_pixiu_data/_data_brussel/100/vsc10009/reframe/eessitestsuite/venv-reframe/lib64/python3.9/site-packages/reframe/core/decorators.py", line 105, in instantiate_all
    leaf_tests.append(test(*args, **kwargs))
  File "/vscmnt/brussel_pixiu_data/_data_brussel/100/vsc10009/reframe/eessitestsuite/venv-reframe/lib64/python3.9/site-packages/reframe/core/meta.py", line 497, in __call__
    obj.__init__(*args, **kwargs)
  File "/vscmnt/brussel_pixiu_data/_data_brussel/100/vsc10009/reframe/eessitestsuite/venv-reframe/lib64/python3.9/site-packages/reframe/core/hooks.py", line 111, in _fn
    getattr(obj, h.__name__)()
  File "/vscmnt/brussel_pixiu_data/_data_brussel/100/vsc10009/reframe/eessitestsuite/venv-reframe/lib64/python3.9/site-packages/reframe/core/hooks.py", line 38, in _fn
    func(*args, **kwargs)
  File "/data/brussel/100/vsc10009/reframe/eessitestsuite/test-suite/eessi/testsuite/eessi_mixin.py", line 159, in EESSI_mixin_run_after_init
    raise ReframeFatalError(err_msg)
reframe.core.exceptions.ReframeFatalError: Invalid thread_binding value 'nonexisting'. Valid values: 'true', 'compact', or 'false'.

to this:

ERROR: run session stopped: eessierror: Invalid thread_binding value 'nonexisting'. Valid values: 'true', 'compact', or 'false'.
ERROR: eessi.testsuite.utils.EESSIError: Invalid thread_binding value 'nonexisting'. Valid values: 'true', 'compact', or 'false'.

casparvl · 2026-02-11T14:11:22Z

Is it really preferable to hide the traceback? I mean, in most cases you just need to respect the suggestion and change it to a valid value. But in some cases, you might want to add support for a new value - and this will essentially provide you a starting point for that.

smoors · 2026-02-11T17:15:45Z

do you have an example in mind?

my idea was that, since the test suite is emitting the error and not ReFrame, the user should not have to see the ugly ReFrame traceback, but we can make exceptions of course.

or we could add an option to increase the traceback limit?

boegel · 2026-02-11T18:55:08Z

@@ -1,5 +1,5 @@
 [build-system]
-requires = ["setuptools>=42", "wheel"]
+requires = ["setuptools>=42,<=81.0.0", "wheel"]


please open an issue on this, mention why you had to start doing this (which errors, etc.)

this should be a temporary measure...

yes, it's the same issue as easybuild: pkg_resources should be replaced

Ah, we're actually using it ourselves in here... https://github.com/search?q=repo%3AEESSI%2Ftest-suite%20pkg_resources&type=code

It's been deprecated since setuptools 67.5.0 (March 2023), see also pypa/setuptools#3085

this may help:

prefer using importlib.metadata over the deprecated pkg_resources in det_pypkg_version + switch container tests to rockylinux 8.10 easybuilders/easybuild-framework#4904

casparvl · 2026-02-12T12:22:34Z

Discussed in the test meeting: we could make this optional, based on a parameter set on the command line (-S something). It would be really nice if the message that is printed then provides this suggestion ("to see the full traceback, rerun with -S ")

smoors · 2026-02-12T18:24:58Z

using a ReFrame variable turned out to be a lot of hassle, so i resorted to an environment variable TRACEBACK

casparvl

Works like a charm. I inserted some test class that didn't properly define the module_name in time and got:

$ reframe -c test-suite/eessi/testsuite/tests/ -t CI -t 1_8_node --system snellius:rome --list
...
ERROR: run session stopped: eessierror: The variable 'module_name' should be defined in any test class that inherits from EESSI_Mixin before (or in) the init phase, but it wasn't
Rerun with `TRACEBACK=1 reframe ...` to show the full traceback.
ERROR: eessi.testsuite.utils.EESSIError: The variable 'module_name' should be defined in any test class that inherits from EESSI_Mixin before (or in) the init phase, but it wasn't
Rerun with `TRACEBACK=1 reframe ...` to show the full traceback.

Then, with TRACEBACK=1:

$ TRACEBACK=1 reframe -c test-suite/eessi/testsuite/tests/ -t CI -t 1_8_node --system snellius:rome --list
...
ERROR: run session stopped: eessierror: The variable 'module_name' should be defined in any test class that inherits from EESSI_Mixin before (or in) the init phase, but it wasn't
ERROR: Traceback (most recent call last):
  File "/gpfs/home4/casparl/EESSI/reframe/reframe/frontend/cli.py", line 1354, in main
    checks_found = loader.load_all(force=True)
  File "/gpfs/home4/casparl/EESSI/reframe/reframe/core/logging.py", line 1116, in _fn
    return fn(*args, **kwargs)
  File "/gpfs/home4/casparl/EESSI/reframe/reframe/frontend/loader.py", line 244, in load_all
    checks += self.load_from_dir(d, self._recurse, force)
  File "/gpfs/home4/casparl/EESSI/reframe/reframe/frontend/loader.py", line 216, in load_from_dir
    checks += self.load_from_dir(entry.path, recurse, force)
  File "/gpfs/home4/casparl/EESSI/reframe/reframe/frontend/loader.py", line 216, in load_from_dir
    checks += self.load_from_dir(entry.path, recurse, force)
  File "/gpfs/home4/casparl/EESSI/reframe/reframe/frontend/loader.py", line 223, in load_from_dir
    checks += self.load_from_file(entry.path, force)
  File "/gpfs/home4/casparl/EESSI/reframe/reframe/frontend/loader.py", line 195, in load_from_file
    return self.load_from_module(util.import_module_from_file(
  File "/gpfs/home4/casparl/EESSI/reframe/reframe/frontend/loader.py", line 156, in load_from_module
    candidate_tests = registry.instantiate_all(reset_sysenv,
  File "/gpfs/home4/casparl/EESSI/reframe/reframe/core/logging.py", line 1116, in _fn
    return fn(*args, **kwargs)
  File "/gpfs/home4/casparl/EESSI/reframe/reframe/core/decorators.py", line 105, in instantiate_all
    leaf_tests.append(test(*args, **kwargs))
  File "/gpfs/home4/casparl/EESSI/reframe/reframe/core/meta.py", line 497, in __call__
    obj.__init__(*args, **kwargs)
  File "/gpfs/home4/casparl/EESSI/reframe/reframe/core/hooks.py", line 111, in _fn
    getattr(obj, h.__name__)()
  File "/gpfs/home4/casparl/EESSI/reframe/reframe/core/hooks.py", line 38, in _fn
    func(*args, **kwargs)
  File "/gpfs/home4/casparl/EESSI/test-suite/eessi/testsuite/eessi_mixin.py", line 145, in EESSI_mixin_validate_init
    raise EESSIError(msg)
eessi.testsuite.utils.EESSIError: The variable 'module_name' should be defined in any test class that inherits from EESSI_Mixin before (or in) the init phase, but it wasn't

Great!

add EESSIError exception to avoid printing traceback

2a8dcbd

Samuel Moors added 2 commits February 11, 2026 19:23

merge main

8505369

restrict setuptools to <=81.0.0

e9a3c3c

boegel reviewed Feb 11, 2026

View reviewed changes

merge main again

5d63213

allow showing the traceback with envvar

ef6d29b

Merge branch 'main' into error

7eb682e

casparvl approved these changes Mar 9, 2026

View reviewed changes

casparvl merged commit 1559cf5 into EESSI:main Mar 9, 2026
16 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

add EESSIError exception to avoid printing traceback#319

add EESSIError exception to avoid printing traceback#319
casparvl merged 6 commits intoEESSI:mainfrom
smoors:error

smoors commented Jan 31, 2026 •

edited

Loading

Uh oh!

casparvl commented Feb 11, 2026

Uh oh!

smoors commented Feb 11, 2026 •

edited

Loading

Uh oh!

boegel Feb 11, 2026

Uh oh!

smoors Feb 11, 2026

Uh oh!

boegel Feb 11, 2026

Uh oh!

boegel Feb 11, 2026

Uh oh!

smoors Feb 12, 2026

Uh oh!

casparvl commented Feb 12, 2026

Uh oh!

smoors commented Feb 12, 2026

Uh oh!

casparvl left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

smoors commented Jan 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

casparvl commented Feb 11, 2026

Uh oh!

smoors commented Feb 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

boegel Feb 11, 2026

Choose a reason for hiding this comment

Uh oh!

smoors Feb 11, 2026

Choose a reason for hiding this comment

Uh oh!

boegel Feb 11, 2026

Choose a reason for hiding this comment

Uh oh!

boegel Feb 11, 2026

Choose a reason for hiding this comment

Uh oh!

smoors Feb 12, 2026

Choose a reason for hiding this comment

Uh oh!

casparvl commented Feb 12, 2026

Uh oh!

smoors commented Feb 12, 2026

Uh oh!

casparvl left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

smoors commented Jan 31, 2026 •

edited

Loading

smoors commented Feb 11, 2026 •

edited

Loading