Add big-request guardrails and honest resource diagnostics - #9669
Open
jhonabreul wants to merge 7 commits into
Open
Add big-request guardrails and honest resource diagnostics#9669jhonabreul wants to merge 7 commits into
jhonabreul wants to merge 7 commits into
Conversation
…f as a corrupt zip
Moves the large/overlapping history request warning state and logic out of the QCAlgorithm partial into a private LargeHistoryRequestDiagnostics nested class with a single entry point, keeping the algorithm class surface small. No behavior change.
Keep the new user-facing warnings and code comments short and direct so they are easy to act on. No behavior change.
jhonabreul
marked this pull request as ready for review
August 12, 2026 17:57
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Adds four independent warn-don't-fail diagnostics for big-request failure modes that today surface as opaque or misleading fatal errors. Runs that would have succeeded are unaffected; all thresholds are config overridable.
ZipDataCacheProvider.Fetch: allocation failures wrapped by Ionic were logged asCorrupt zip file/entryand swallowed. AnOutOfMemoryExceptionanywhere in the chain is now rethrown as a memory diagnostic. Truly corrupt zips keep the old behavior.QCAlgorithm.Historyfunnel, so all overloads): warns once over an estimated data-cell threshold (default 5M), and warns once when 30 consecutive large calls have overlapping time windows (the re-fetch-a-long-lookback-every-day pattern).AlgorithmTimeLimitManagerlogs a warning once per time step over a threshold (default 1 minute) instead of staying silent until the isolator kill, andTimeMonitornow names the running scheduled event when it crosses the one-minute mark.OptionChain()+AddOptionContract.Related Issue
N/A
Motivation and Context
These failure modes cost hours: a wide option universe dying as "corrupt zip" (actually OOM), billion-cell history re-fetches stalling backtests, and 10-minute isolator kills that never name the slow handler. Upfront, named warnings make them self-correcting.
Requires Documentation Change
No. The new config keys only tune warnings and have safe defaults.
How Has This Been Tested?
AlgorithmHistoryTestsfixture.BasicTemplateOptionsAlgorithm(C# + Python) statistics unchanged.Types of changes
Checklist:
bug-<issue#>-<description>orfeature-<issue#>-<description>