Skip to content

Improve ObjectStore ergonomics for end-of-run persistence - #9674

Draft
jhonabreul wants to merge 1 commit into
QuantConnect:masterfrom
jhonabreul:feature-object-store-ergonomics
Draft

Improve ObjectStore ergonomics for end-of-run persistence#9674
jhonabreul wants to merge 1 commit into
QuantConnect:masterfrom
jhonabreul:feature-object-store-ergonomics

Conversation

@jhonabreul

@jhonabreul jhonabreul commented Aug 11, 2026

Copy link
Copy Markdown
Collaborator

Description

End-of-run persistence to the ObjectStore is a common pattern, and its most frequent failures surface only in OnEndOfAlgorithm, losing the output of an otherwise green run:

LocalObjectStore: path is not supported: 'trade_log_ai_hardware_&_cloud.csv'
Object of type datetime is not JSON serializable
'ObjectStore' object has no attribute 'save_text' Did you mean: 'save'?

The changes:

  • ObjectStore.IsSupportedKey / SanitizeKey (static): key validation moves to the ObjectStore wrapper as the single source; SanitizeKey converts an arbitrary name into a supported key.
  • Save rejection errors now state the full key rules and point to SanitizeKey.
  • Reading a missing key now lists the available keys (first 10), so typos are visible in the error.
  • SaveJson(path, PyObject) / ReadJson(path, default=None): Python JSON helpers with a tolerant encoder (datetime/date as ISO-8601, Decimal/numpy as numbers, Symbol and other types via str()). The generic C# overloads are untouched.
  • SaveDataframe(path, dataFrame): stores a pandas DataFrame/Series as CSV (JSON for .json keys).
  • SaveText: alias of Save(path, text, encoding).

Related Issue

N/A

Motivation and Context

Completed backtests can die on their final persistence call. Eager, self-explanatory validation plus tolerant JSON turns these total-loss failures into first-try successes.

Requires Documentation Change

Yes: the new ObjectStore members and the improved errors.

How Has This Been Tested?

  • New LocalObjectStoreTests cases: key validation, rejection message, SanitizeKey cases, missing-key listing, SaveText, plus a Python-side test for save_json/read_json/save_dataframe. Full storage suite: 136 passed, 0 failed.
  • New ObjectStoreErgonomicsRegressionAlgorithm (C# + Python): rejection, sanitize-then-save, text round trip, missing-key error; the Python twin adds the tolerant JSON round trip and save_dataframe. Both pass.
  • Reproduced all failure signatures through the Launcher before the change; each now succeeds or produces the self-explanatory error.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • Refactor (non-breaking change which improves implementation)
  • Performance (non-breaking change which improves performance. Please add associated performance test and results)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Non-functional change (xml comments/documentation/etc)

Checklist:

  • My code follows the code style of this project.
  • I have read the CONTRIBUTING document.
  • I have added tests to cover my changes.
  • All new and existing tests passed.
  • My branch follows the naming convention bug-<issue#>-<description> or feature-<issue#>-<description>

- Key rules (charset/extension) move to static ObjectStore.IsSupportedKey;
  unsupported-key errors now state the rules and point to the new
  ObjectStore.SanitizeKey helper for programmatically-built keys
- Reading a missing key lists the available keys in the error
- New Python-facing helpers: save_json/read_json with a tolerant encoder
  (datetime/date/time, Decimal, numpy scalars, Symbol, non-string dict keys)
  and read_json default for missing keys, save_dataframe (CSV, or JSON for
  .json keys) and save_text as an alias of save
- Unit tests plus ObjectStoreErgonomicsRegressionAlgorithm (C# + Python)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant