feat(producer): optional targetChunkFrames to bound per-chunk frames

[jrusso1020]

Summary

Adds an optional targetChunkFrames knob to DistributedRenderConfig that caps frames-per-chunk in the auto-sized chunking path. Today the two chunking knobs are alternatives: maxParallelChunks caps the chunk count (so per-chunk frames — and thus per-chunk render time — grow unboundedly with video length), and chunkSize pins a fixed size (so short videos can't collapse to fewer chunks). Neither lets a caller say "size chunks to ~N frames each, but never more than M chunks." targetChunkFrames is that third option:

chunkCount = clamp(ceil(totalFrames / targetChunkFrames), 1, maxParallelChunks)

• Short videos collapse to fewer chunks (less per-chunk fixed overhead — boot, plan download, planHash recompute, ffmpeg init).
• Long videos add chunks, up to maxParallelChunks, to keep each chunk's render time under a downstream per-chunk timeout.
• It's a ceiling, not a fixed size: a video short enough to fit in fewer chunks gets fewer.

This is the one-pass equivalent of the "plan, observe frame count, re-plan at a higher fan-out" dance a caller otherwise has to do in orchestration when a chunk-count cap leaves a long video with chunks too big to finish in time.

Default behavior is unchanged

targetChunkFrames is optional and only affects the auto-sized path (chunkSize === undefined):

• Omitted (or undefined) → autoSizeParallel === maxParallelChunks → the auto-sized chunk size is identical to prior behavior. A test asserts the 3-arg and 4-arg-undefined results are byte-for-byte equal across a grid of inputs.
• chunkSize set → targetChunkFrames is a no-op (an explicit fixed size already pins per-chunk frames; chunkSize wins). A test asserts equality with the chunkSize-only result.
• New optional 4th param on resolveChunkPlan and new optional field on the config type — existing 3-arg callers and existing serialized configs are unaffected.

Validation mirrors the existing chunkSize/maxParallelChunks checks (positive integer, <= MAX_CHUNK_SIZE) in the shared validateDistributedRenderConfig, which both the AWS Lambda and GCP Cloud Run adapters already delegate to — so the bound applies everywhere with no adapter changes.

Testing

• tsc --noEmit clean; oxlint + oxfmt clean.
• resolveChunkPlan suite: 18 pass, including 7 new cases — omitted-is-a-no-op (grid), short-collapse, long-video bounding, tier-cap clamp (extreme length stays at the cap), chunkSize-wins precedence, non-positive/non-integer rejection, and an empty/inverted-slice grid over targetChunkFrames.
• publicExports + planFormatBanlist: 10 pass. AWS Lambda + GCP Cloud Run validateConfig suites: 51 pass (confirming the additive field doesn't disturb existing callers).

Context

This is the clean primitive behind a chunking policy HeyGen currently implements as a post-plan re-plan in its orchestration; once this ships and the producer build is deployed, that re-plan collapses to passing targetChunkFrames + maxParallelChunks on a single plan call. Useful standalone for any distributed-render caller who wants to bound per-chunk runtime on long videos.

🤖 Generated with Claude Code

[miguel-heygen]

Clean feature addition. Math is correct across all cases; test coverage is solid.

Traced the formula manually:

• Short video (1466f, target 300, cap 16): autoSizeParallel=5, effectiveChunkSize=294 ≤ 300 ✓
• Long video (54000f, target 1600, cap 64): autoSizeParallel=34, effectiveChunkSize=1589 ≤ 1600 ✓
• Extreme clamp (216000f, target 1600, cap 64): clamps to 64 chunks, effectiveChunkSize=3375 > 1600 (documented behavior) ✓
• No-op when omitted: autoSizeParallel === maxParallelChunks, identical to prior path ✓

P3 nits:

1. Inconsistent upper-bound validation: validateDistributedRenderConfig checks targetChunkFrames > MAX_CHUNK_SIZE, but resolveChunkPlan only calls assertPositiveInteger (no upper-bound check). chunkSize has the same pattern — upper-bound only in the config validator — so this is consistent with the existing convention, but direct callers of resolveChunkPlan bypass the ceiling. Document-only risk since the config validator gates the public API.

2. Silent no-op when chunkSize and targetChunkFrames are both set: documented in the JSDoc ("Ignored when chunkSize is set"), but the validator doesn't warn. A user who mistakenly sets both will silently get chunkSize behavior with no feedback. A validator warning would save confusion — not required now but worth a follow-up.

3. No dedicated test for targetChunkFrames >= totalFrames (single-chunk collapse): e.g., 100 frames, target 300 → should produce 1 chunk. The "empty/inverted slice" grid gets close (37 frames, target 100 → 1 chunk, no slice assertion fails), but no explicit chunkCount === 1 assertion for this edge. Fine for now.

Approving.

[vanceingalls]

Review of #1332 — optional targetChunkFrames to bound per-chunk frames

Verdict: APPROVE. Tight design, additive change, sensible defaults, generous test grid. One nit on a doc-vs-behavior interaction the author may want to acknowledge.

What I checked

• Walked resolveChunkPlan against the grid the test asserts (50/660/1466/12793/54000/216000 × parallels 8/16/64 × targets 100/300/1600).
• Confirmed assertPositiveInteger, the MIN_CHUNK_SIZE floor, and the trailing-slice-tightening pass interact correctly with the new branch.
• Verified validator bound (MAX_CHUNK_SIZE = 3600) is semantically the right one — targetChunkFrames caps frames-per-chunk, same unit and meaning as chunkSize.
• Traced the new optional field through validateDistributedRenderConfig and the plan() call site. Adapters that JSON-pass-through event.Config (packages/aws-lambda, packages/gcp-cloud-run) propagate it transparently because they spread {...event.Config} rather than enumerating fields.
• Confirmed SerializableDistributedRenderConfig is Omit<DistributedRenderConfig, ...> so the new optional field is automatically included for the JSON-wire shape; existing serialized configs without the field still parse (optional, === undefined gate).

Findings

[nit] Doc claims "bounds per-chunk render time" but MIN_CHUNK_SIZE = 10 silently overrides on very small targets.

The validator accepts targetChunkFrames >= 1. Trace: with totalFrames = 50, targetChunkFrames = 5, maxParallelChunks = 16:

• autoSizeParallel = min(16, max(1, ceil(50/5))) = 10
• resolvedChunkSize = max(MIN_CHUNK_SIZE=10, ceil(50/10)=5) = 10
• final result: 5 chunks of 10 frames each — silently 2× over the requested ceiling.

In practice operators set targetChunkFrames in the hundreds-to-thousands range (timeout-bounding for cloud-run/lambda), so this is academic. But the doc on DistributedRenderConfig.targetChunkFrames reads "chunking targets the fewest chunks whose per-chunk frame count stays at or below this bound" — strictly false when target < MIN_CHUNK_SIZE. Two ways to close: (a) validator additionally rejects targetChunkFrames < MIN_CHUNK_SIZE with a message pointing at the floor, or (b) the doc-comment names the interaction (something like "the auto-size floor MIN_CHUNK_SIZE still applies, so targetChunkFrames < MIN_CHUNK_SIZE is effectively pinned at the floor"). I'd take (a) — quieter contract, no surprises for callers wiring up a config builder.

Things that look right (not for change)

• autoSizeParallel = min(maxParallelChunks, max(1, ceil(totalFrames / targetChunkFrames))) is the cleanest expression of the clamp; the inner Math.max(1, …) is technically dead given assertPositiveInteger upstream but cheap and defensive against future refactors.
• Precedence (chunkSize wins over targetChunkFrames) is documented, tested, and matches operator mental model — explicit per-chunk size already pins the dimension.
• Validator order matches the existing pattern: positive-int check, then upper bound. Same error shape as chunkSize/maxParallelChunks.
• The "never emits an empty or inverted slice across a grid" test is the right correctness invariant. The 216000-frame "stays at the cap, exceeds target" test is a particularly nice addition — it pins the documented escape hatch behavior rather than silently changing it later.
• targetChunkFrames=2000, totalFrames=1466 (target > totalFrames) collapses to a single chunk of 1466 frames; covered by the no-op test on the principle and by the 1466/300 short-video case.
• Schema: optional field, gated by !== undefined, default branch is byte-identical to prior behavior (test confirms). The PR description's "existing serialized configs are unaffected" is provably true.

— Review by Vai

[jrusso1020]

Folded in CLI exposure + docs so the knob is usable end-to-end, not library-only (commit 7cf9b21):

• CLI flag --target-chunk-frames on hyperframes lambda render / render-batch and hyperframes cloudrun render — sits beside the existing --chunk-size / --max-parallel-chunks, parsed via the same parsePositiveInt and threaded into the distributed config.
• Docs: docs/packages/cli.mdx (cloudrun render flag list), docs/deploy/migrating-to-hyperframes-lambda.mdx (flags table — new per-chunk-frame-ceiling row), docs/deploy/templates-on-lambda.mdx (chunking cost-knobs section, with the long-video-timeout rationale + the tier-cap-clamp caveat).

Testing: CLI package tsc --noEmit clean; oxlint + oxfmt clean on the four command files. The command-suite failures on this host (render-batch.test.ts JSON-parse, layout-audit.browser, skills, runAdd) reproduce identically with these changes stashed — pre-existing host/fixture issues, not introduced here. The resolveChunkPlan suite (18) still passes.