[Metrics] Add preemption metrics and accumulation logic - #5396
Conversation
1064ba3 to
f369557
Compare
f369557 to
0c5f861
Compare
| ], | ||
| ) | ||
|
|
||
| FUZZER_PREEMPTED_TOTAL_FUZZ_TIME = monitor.CounterMetric( |
There was a problem hiding this comment.
This doesn't include build set up and corpus sync time, which is also effectively wasted time from a fuzzing perspective. But our total fuzz time metrics also don't include that, so I understand that we may want it to match.
I don't know how long that set up takes, but it might be worth tracking that as well.
There was a problem hiding this comment.
Also, not blocking at all because any metrics here will help but definitely worth discussing/thinking about
There was a problem hiding this comment.
Yes, it was done that way to match the existing metric, so the subtraction (total_time - preempted_time) makes sense.
Measuring the duration of the full main stage (including build set up and corpus sync) is the next step I'm working on. Thanks for pointing that out!
b/538558303
Note: this PR is chained with #5394
Problem
When a preemption occurs, we were only recording the duration of the active fuzzing round as wasted time. If a task is preempted mid-way, all previous successful rounds in that same task are also completely wasted since the task never reaches postprocess. This leads to a significant underestimation of wasted time.
Proposed Solution
Enhanced time tracking to capture the waste across the entire fuzzing session while maintaining metric accuracy.
FUZZER_PREEMPTED_TOTAL_FUZZ_TIMEandJOB_PREEMPTED_TOTAL_FUZZ_TIMEmetrics tomonitoring_metrics.py._TrackFuzzTimeinfuzz_task.pyto track accumulated wasted time across all rounds of a session (_accumulated_wasted_time).preempted_total_time.total_time, ensuring it reflects the attempted time without gaps.tracker.preemptedflag to skip metric emission in__exit__for interrupted rounds and avoid double counting.Validation
Validated in dev environment.