Dev: Ensure tests are run in random

[paulbalandan]

PHP Version

8.5.0

CodeIgniter4 Version

4.7.1-dev

Per ChatGPT, tests need to run in random to enjoy the following benefits:

✔️ It reveals hidden dependencies
✔️ Forces true test isolation
✔️ Prevents flaky, misleading test results
✔️ Makes your test suite production-realistic

This issue has been initially tackled by #7717, but due to other complications it didn't fare well. As this is not a fairly easy task, I'm proposing we do it on pieces. Currently, our test suite runs in default order but should be random at some point (i.e., executionOrder="random").

Here's the plan, instead of one big PR changing all tests, we can do it either by component (HTTP, Cache, Database, etc.). Then, after all tests run green on randomised testing, we set executionOrder in phpunit.xml to "random".

Please reference this issue for any PRs addressing pieces of this.

• API
• Autoloader
• Cache
• CLI
• Commands
• Config
• Cookie
• Database
• DataCaster
• DataConverter
• Debug
• Email
• Encryption
• Entity
• Events
• Exceptions
• Files
• Filters
• Format
• Helpers
• Honeypot
• HotReloader
• HTTP
• I18n
• Images
• Language
• Log
• Modules
• Pager
• Publisher
• RESTful
• Router
  • test: fix determinism of CacheTest #9967
• Security
  • refactor: fix Security test fail by itself #9969
• Session
• Test
• Throttle
• Typography
• Validation
• View

[neznaika0]

I also mentioned the randomization of tests, the answer was the same - it takes a lot of time.

I can't imagine how to edit Database tests, this is the most difficult stage.

[paulbalandan]

I also think randomising database tests would be the most difficult piece.