GD-995: Allow creating of tests that use editor-only classes

[derkork]

Feature Type

Test runner enhancement

Problem/Use Case

I am writing a lot of editor extensions for Godot, and right now, there is no way to have automated tests for editor extensions with GDUnit because all the tests run inside of a Godot test running instance which does not have access to editor-only classes.

I'm not sure if this is actually doable, but it would be nice if one could specify that a test is an in-editor test. In-editor-tests either run against the current editor instance or maybe against a new editor instance that is spawned up like the test running instance. It may be a bit heavy to spawn up a new editor instance, though.

This is more on the nice-to-have sort of things, but I'm leaning more into automated tests these days. Being able to test and write automated tests for add-on editor functionality would be really, really helpful.

Proposed Solution

The simplest thing that I can think of is just make another base class for tests that should run with editor APIs enabled:

extends GdUnitEditorTestSuite

... tests as usual...

Alternatives Considered

Right now I don't have any alternatives. I'm testing my stuff manually, which isn't really great because it takes a lot of time to do, especially if you have more complex editor integrations.

Impact and Benefits

This feature would help people who create editor integrations for Godot, either for themselves or for the asset library/asset store. It is, of course, not as common as creating tests for actual games, but it is still a thing that would probably have a wide range of users. As far as I know, there's currently no testing framework for Godot that would support this.

Implementation Notes

I think starting up a new editor on a dummy project before running this would probably be better, but I can already see how this is going to add a whole ton of headaches because you would need to register the editor classes in this dummy project. Also, starting up a new editor is probably not exactly a cheap operation, so it could introduce quite a bit of round-trip time. Because automated tests are actually intended to reduce the round-trip time that would probably be counter-productive. That leaves running these tests directly in the current editor instance, which may be okay if they are not destructive. But, of course, you would run into the problem that you could destroy your project by a rogue test.

Examples and References

As far as I know, currently no Godot testing framework supports this, so there's no examples for this.

Contribution

• I'm willing to contribute code for this feature
• I'm willing to help with testing/feedback
• I'm willing to help with documentation
• I would just like to request this feature

[MikeSchulze]

Hello @derkork, this is a hot topic.

This feature has been requested several times and cannot be implemented easily.
This is because plugin instances, such as EditorInterface, are only accessible via loaded plugins directly through the editor instance.

Since the tests are ultimately executed via a scene and these do not have access to internal editor instances, this is currently not possible.

The test runner would have to be integrated directly into the internal editor main tree so that static calls such as EditorInterface.get_script_editor() also have access to the instances.

Another problem is the CI runner, where ultimately only a “game” scene without editor functionality is played.

[derkork]

Yeah I can see how this is difficult to implement as the tests would basically need to run in an editor instance not just a runtime instance. The whole launching setup would need to be different. I currently don't have a good idea on how this could be done, so i guess we'll have to put this one on the backburner for now.

[AnidemDex]

Just wanted to add 2 cents comment: maybe you can launch an editor instance on runtime to test the scripts?
godotengine/godot#5983 (comment)
godotengine/godot#109665

For now is the way that I've been debugging my Editor* classes

[MikeSchulze]

Thanks for providing this informations 👍
I did the same trick with the GdScript code coverage plugin i'm actual writing ;)