test: Enable test_snapshot_pruning_removes_outdated_records#1735
test: Enable test_snapshot_pruning_removes_outdated_records#1735
test_snapshot_pruning_removes_outdated_records#1735Conversation
… expected source of flaky behavior
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #1735 +/- ##
==========================================
- Coverage 92.50% 92.42% -0.08%
==========================================
Files 156 156
Lines 10602 10602
==========================================
- Hits 9807 9799 -8
- Misses 795 803 +8
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
vdusek
left a comment
There was a problem hiding this comment.
FAILED tests/unit/_autoscaling/test_snapshotter.py::test_snapshot_pruning_removes_outdated_records - assert 3 == 2
- where 3 = len([CpuSnapshot(used_ratio=0.5, max_used_ratio=0.95, created_at=datetime.datetime(2026, 2, 11, 7, 39, 32, 789908, tzinfo=datetime.timezone.utc)), CpuSnapshot(used_ratio=0.5, max_used_ratio=0.95, created_at=datetime.datetime(2026, 2, 11, 9, 39, 32, 789908, tzinfo=datetime.timezone.utc)), CpuSnapshot(used_ratio=0.5, max_used_ratio=0.95, created_at=datetime.datetime(2026, 2, 11, 10, 39, 32, 789908, tzinfo=datetime.timezone.utc))])
============ 1 failed, 1615 passed, 8 skipped in 345.52s (0:05:45) =============
Error: Process completed with exit code 1.
d7783c5 to
63e8468
Compare
|
@Mantisus , @vdusek, please check the root cause in the issue description. The suggested fix should be ok, but I fear it is too implicit and relies on |
The root cause hypothesis seems incorrect. If the cause was an error processing in the 2-hour snapshot, then there would be 1 snapshot in the failed tests, not 3. It might be worth trying for event_data in events_data:
event_manager.emit(event=Event.SYSTEM_INFO, event_data=event_data)
await event_manager.wait_for_all_listeners_to_complete()Perhaps the reason is that |
Description:
test_snapshot_pruning_removes_outdated_recordsto remove the expected source of flaky behaviorevent_manager.on(event=Event.SYSTEM_INFO, listener=self._snapshot_cpu)addsself._snapshot_cpulistener, but event manager runs sync listeners throughasyncio.to_thread.self._snapshot_cpumodifies in-place instance list (for exampleself._cpu_snapshots) - it doesbisect.insortanddeloperations on the same list from several threads, which creates oportunity for a race condition.Issues:
Closes: #1734