fix(tests): fix vpc model tests to use robust Json comparison#78
Open
Fr4nk03 wants to merge 1 commit intoIBM:masterfrom
Open
fix(tests): fix vpc model tests to use robust Json comparison#78Fr4nk03 wants to merge 1 commit intoIBM:masterfrom
Fr4nk03 wants to merge 1 commit intoIBM:masterfrom
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem:
This PR resolves intermittent test failures observed across various VPC model tests, using the NonDex tool.
https://github.com/TestingResearchIllinois/NonDex
In the pull request #38, the contributor didn't explain the root cause of the problem.
Root Cause
The toString() method on the SDK models relies on Gson serialization to produce a JSON string.
@Override public String toString() { return GsonSingleton.getGson().toJson(this); }from https://github.com/IBM/java-sdk-core/blob/main/src/main/java/com/ibm/cloud/sdk/core/service/model/GenericModel.java.Gson uses Java Reflection and getDeclaredFields() to determine the fields in the object. However,
This issue was also raised in the gson repository: https://github.com/google/gson/issues/617
Hence, in some JVM environments, the reflection API returns object fields in a non-deterministic (random) order. Simple string assertEquals fails in this scenario, leading to intermittent test failures.
Solution: Canonical JSON Comparison
To fix this, I have introduced a robust utility method, TestUtilities.assertJsonEquals(), and applied it to all affected model tests.
This new utility implements a canonical comparison strategy:
Comparison with JsonParser.parseString:
The fix in this Pull Request uses TreeMap .equals() comparison whereas JsonParser uses JsonElement .equals() comparison. Both fixes ignores iteration order and compares content, ensuring test stability across all environments.