At present we test that the recipes run without error, but output validity checking is left to eyeballing the rendered recipes. To provide a more robust test that the outputs, which are often plots of some form, are as expected and notably haven't changed from a previous output unless that is expected (so as a form of regression testing), we should provide image comparison tests.
I have implemented these in cf-plot based on helper utilities provided in matplotlib for such testing (see NCAS-CMS/cf-plot#13 and NCAS-CMS/cf-plot#14), so a solution can be added fairly easily (I anticipate, at least) based on that.
This work is best left until the recipes have been assigned a separate build i.e. after #916 is closed.
At present we test that the recipes run without error, but output validity checking is left to eyeballing the rendered recipes. To provide a more robust test that the outputs, which are often plots of some form, are as expected and notably haven't changed from a previous output unless that is expected (so as a form of regression testing), we should provide image comparison tests.
I have implemented these in cf-plot based on helper utilities provided in matplotlib for such testing (see NCAS-CMS/cf-plot#13 and NCAS-CMS/cf-plot#14), so a solution can be added fairly easily (I anticipate, at least) based on that.
This work is best left until the recipes have been assigned a separate build i.e. after #916 is closed.