Skip to content

[OMNIML-5562] Add FAR3D ONNX PTQ and accuracy evaluation example - #2012

Open
ajrasane wants to merge 10 commits into
mainfrom
ajrasane/onnx_eval
Open

[OMNIML-5562] Add FAR3D ONNX PTQ and accuracy evaluation example#2012
ajrasane wants to merge 10 commits into
mainfrom
ajrasane/onnx_eval

Conversation

@ajrasane

@ajrasane ajrasane commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

What does this PR do?

Type of change: new example

Adds an end-to-end FAR3D ONNX PTQ example under examples/onnx_ptq/far3d. The example prepares Argoverse 2 validation metadata and calibration batches, quantizes the image encoder to INT8, builds TensorRT engines, and evaluates 3D object detection accuracy.

It also provides a reproducible FAR3D runtime image, preserves accuracy-sensitive encoder layers in high precision, and supports temporal decoder state during evaluation.

Usage

python prepare_metadata.py /path/to/av2
python prepare_calibration.py /path/to/far3d.py far3d_calibration
python quantize.py far3d.encoder.onnx far3d_calibration
python evaluate.py /path/to/far3d.py \
  far3d.encoder.int8.engine far3d.decoder.fp16.engine

Testing

  • Ran all configured pre-commit hooks on the changed files.
  • Ran synthetic calibration-reader and graph-exclusion tests.
  • Built the documented FAR3D runtime image and verified TensorRT 10.11, Argoverse 2 imports, and TensorRT engine execution.
  • Ran the complete workflow on the Argoverse 2 validation split using 500 calibration batches and 23,522 evaluation frames. The INT8 encoder and FP16 decoder produced 0.238 mAP.

Before your PR is "Ready for review"

Make sure you read and follow Contributor guidelines and your commits are signed (git commit -s -S).

Make sure you read and follow the Security Best Practices (e.g. avoiding hardcoded trust_remote_code=True, torch.load(..., weights_only=False), pickle, etc.).

  • Is this change backward compatible?: ✅
  • If you copied code from any other sources or added a new PIP dependency, did you follow guidance in CONTRIBUTING.md: ✅
  • Did you write any new necessary tests?: N/A
  • Did you update Changelog?: N/A

Additional Information

🤖 Generated by Codex (AI agent).

Summary by CodeRabbit

  • New Features
    • Added an end-to-end FAR3D ONNX post-training quantization workflow for Argoverse 2, including calibration batch generation, INT8/FP8 quantization, TensorRT engine inference, and mAP evaluation.
    • Added a dedicated FAR3D example Docker environment plus detailed README instructions.
    • Added FAR3D evaluation, calibration preparation, metadata preparation, and quantization scripts.
  • Bug Fixes
    • Improved handling when flash-attention is unavailable, with clearer error messaging.
  • Chores
    • Updated pre-commit exclusions and refreshed third-party license attribution.
    • Added an Experimental changelog entry for the FAR3D example.

Co-Authored-By: Codex <codex@openai.com>
Signed-off-by: ajrasane <131806219+ajrasane@users.noreply.github.com>
@copy-pr-bot

copy-pr-bot Bot commented Jul 23, 2026

Copy link
Copy Markdown

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

@coderabbitai

coderabbitai Bot commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

Adds a FAR3D ONNX post-training quantization example with a CUDA container, Argoverse 2 metadata and calibration preparation, INT8/FP8 quantization, TensorRT engine evaluation, and workflow documentation.

Changes

FAR3D ONNX PTQ workflow

Layer / File(s) Summary
FAR3D environment and workflow setup
.pre-commit-config.yaml, LICENSE, CHANGELOG.rst, examples/onnx_ptq/README.md, examples/onnx_ptq/far3d/Dockerfile, examples/onnx_ptq/far3d/README.md, examples/onnx_ptq/far3d/far3d_pytorch_25_06.patch
Adds the CUDA/Torch environment, FAR3D setup and execution instructions, FlashMHA import handling, OpenMMLab attribution, license-hook exclusion, and changelog entry.
Dataset metadata and calibration generation
examples/onnx_ptq/far3d/prepare_metadata.py, examples/onnx_ptq/far3d/prepare_calibration.py
Generates Argoverse 2 validation metadata and writes encoder .npy and decoder .npz calibration batches.
Encoder and decoder ONNX quantization
examples/onnx_ptq/far3d/quantize.py
Adds calibration readers, encoder accuracy-sensitive node exclusions, INT8/FP8 quantization, and optional FP16 decoder processing.
TensorRT inference and evaluation
examples/onnx_ptq/far3d/evaluate.py
Adds TensorRT execution, stateful decoder processing, FAR3D input unpacking, prediction conversion, and dataset metric evaluation.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant ValidationDataloader
  participant Far3DPipeline
  participant TensorRTRunner
  participant Far3DDecoderRunner
  participant DatasetEvaluate
  ValidationDataloader->>Far3DPipeline: dataset batch
  Far3DPipeline->>TensorRTRunner: run encoder engine
  TensorRTRunner-->>Far3DPipeline: encoder outputs
  Far3DPipeline->>Far3DDecoderRunner: run decoder with frame metadata
  Far3DDecoderRunner->>TensorRTRunner: execute stateful decoder engine
  TensorRTRunner-->>Far3DDecoderRunner: decoder outputs and state tensors
  Far3DDecoderRunner-->>Far3DPipeline: predictions
  Far3DPipeline->>DatasetEvaluate: prediction records
  DatasetEvaluate-->>Far3DPipeline: dataset metrics
Loading

Suggested reviewers: kevalmorabia97

🚥 Pre-merge checks | ✅ 5 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately summarizes the main change: a new FAR3D ONNX PTQ example with accuracy evaluation.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Security Anti-Patterns ✅ Passed The only changed file is evaluate.py, and it contains none of the banned patterns or dependency-manifest changes.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch ajrasane/onnx_eval

Comment @coderabbitai help to get the list of available commands.

@codecov

codecov Bot commented Jul 23, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 75.74%. Comparing base (d984de3) to head (7988f96).
⚠️ Report is 4 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2012      +/-   ##
==========================================
- Coverage   78.47%   75.74%   -2.73%     
==========================================
  Files         518      518              
  Lines       58574    58574              
==========================================
- Hits        45967    44368    -1599     
- Misses      12607    14206    +1599     
Flag Coverage Δ
examples 43.30% <ø> (-0.20%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

ajrasane and others added 3 commits July 23, 2026 23:55
Co-Authored-By: Codex <codex@openai.com>
Signed-off-by: ajrasane <131806219+ajrasane@users.noreply.github.com>
Co-Authored-By: Codex <codex@openai.com>
Signed-off-by: ajrasane <131806219+ajrasane@users.noreply.github.com>

@cjluo-nv cjluo-nv left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bot review (bedrock-claude-opus-4-8) — DM the bot to share feedback.

New end-to-end FAR3D ONNX PTQ + Argoverse 2 eval example under examples/onnx_ptq/far3d/ (+923/-0, 10 files). The modelopt.onnx.quantization.quantize API usage checks out (verified args: quantize_mode, calibration_data_reader, calibration_eps, nodes_to_exclude, high_precision_dtype, output_path all match the current signature), the graph-exclusion BFS in find_encoder_nodes_to_exclude is sound ("downstream of lateral_convs" + all OSA4_5 nodes), and reusing modelopt.onnx.quantization.quantize rather than reinventing quantization is the right call (no duplicate subsystem introduced — the new code is example glue: TRT runners, calibration readers, dataset plumbing). No prompt-injection attempts in the untrusted content.

Flagging for human sign-off rather than approving because:

  1. Licensing — requires human review (cannot auto-approve). This PR (a) edits the top-level LICENSE (adds Copyright (c) OpenMMLab. All rights reserved. under the Apache-2.0 third-party section) and (b) copies/adapts external code: evaluate.py is "Adapted from NVIDIA/DL4AGX … modified from megvii-research/Far3D", carrying an OpenMMLab copyright + "Modified by Zhiqi Li" stacked above the standard NVIDIA SPDX header. It also vendors a .patch that modifies third-party FAR3D source. The attribution looks conscientious (correct license bucket in LICENSE, retained upstream notices), but license decisions have legal implications and need a human to sign off. Please confirm the combined third-party+NVIDIA header on evaluate.py and the DL4AGX/Far3D provenance are acceptable per CONTRIBUTING.md.

  2. No tests, and no CI-exercisable path. Author marks tests N/A and the PR body mentions "synthetic calibration-reader and graph-exclusion tests" that aren't in the diff. Consistent with the repo's example-dir convention (no other onnx_ptq example ships unit tests), and the entire workflow is GPU/TensorRT/mmdet3d-only so it can't run in CPU CI. The one piece of self-contained, testable logic (find_encoder_nodes_to_exclude) could reasonably get a small unit test if the team wants any coverage.

  3. CHANGELOG not updated (marked N/A). New examples are often exempt, but per repo convention new features get a CHANGELOG bullet — owner's call whether this example warrants one.

Co-Authored-By: Codex <codex@openai.com>
Signed-off-by: ajrasane <131806219+ajrasane@users.noreply.github.com>
@ajrasane
ajrasane marked this pull request as ready for review July 24, 2026 20:03
@ajrasane
ajrasane requested review from a team as code owners July 24, 2026 20:03
Co-Authored-By: Codex <codex@openai.com>
Signed-off-by: ajrasane <131806219+ajrasane@users.noreply.github.com>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Warning

CodeRabbit couldn't request changes on this pull request because it doesn't have sufficient GitHub permissions.

Please grant CodeRabbit Pull requests: Read and write permission and re-run the review.

👉 Steps to fix this

Actionable comments posted: 1

🧹 Nitpick comments (4)
examples/onnx_ptq/far3d/evaluate.py (2)

257-260: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Move the conditional mmcv.utils import to the top-level imports.

This local import has no explanatory comment, and mmcv (the same package) is already imported at the top of the file, so it isn't a genuinely circular, optional, or heavy dependency case.

As per coding guidelines: "Keep imports at the top of Python source and test files; use local imports only for justified circular dependencies, optional dependencies, or unusually heavy imports, with a brief explanatory comment."

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@examples/onnx_ptq/far3d/evaluate.py` around lines 257 - 260, Move
import_modules_from_strings from mmcv.utils into the top-level import section of
evaluate.py, alongside the existing mmcv import, and remove the local import
inside the cfg.get("custom_imports") conditional. Preserve the conditional
invocation and its existing arguments.

Source: Coding guidelines


59-142: 🚀 Performance & Scalability | 🔵 Trivial | ⚡ Quick win

Consider pre-allocating and reusing I/O buffers instead of reallocating on every call.

input_shapes/output_shapes are fixed once in __init__ (static engine shapes) and never change, yet __call__ calls aligned_tensor(...) for every input/output on every invocation — across the full 23,522-frame evaluation (encoder + decoder) plus calibration prep. self.state already demonstrates the allocate-once/reuse pattern; the same approach could apply to the non-state input/output buffers, avoiding repeated CUDA allocations on this hot path.

♻️ Sketch of the refactor
     def __init__(self, engine_path, state_names=()):
         ...
+        self.input_buffers = {
+            name: aligned_tensor(shape, self.tensor_dtypes[name], "cuda")
+            for name, shape in self.input_shapes.items()
+            if name not in self.state
+        }
+        self.output_buffers = {
+            name: aligned_tensor(shape, self.tensor_dtypes[name], "cuda")
+            for name, shape in self.output_shapes.items()
+        }
+        for name, buf in {**self.input_buffers, **self.output_buffers}.items():
+            self.context.set_tensor_address(name, buf.data_ptr())

     def __call__(self, stream, **inputs):
-        input_buffers = {}
         for name, shape in self.input_shapes.items():
             if name in self.state:
                 continue
             value = self.prepare_input(name, inputs)
-            buffer = aligned_tensor(shape, value.dtype, value.device)
-            buffer.copy_(value)
-            input_buffers[name] = buffer
-            self.context.set_tensor_address(name, buffer.data_ptr())
-
-        outputs = {}
-        for name, shape in self.output_shapes.items():
-            output = aligned_tensor(shape, self.tensor_dtypes[name], "cuda")
-            outputs[name] = output
-            self.context.set_tensor_address(name, output.data_ptr())
+            self.input_buffers[name].copy_(value)
 
         if not self.context.execute_async_v3(stream.cuda_stream):
             raise RuntimeError("TensorRT execution failed")
         stream.synchronize()
-        return outputs
+        return self.output_buffers
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@examples/onnx_ptq/far3d/evaluate.py` around lines 59 - 142, Pre-allocate
reusable non-state input and output buffers in TensorRTRunner.__init__, using
the fixed input_shapes, output_shapes, and tensor_dtypes, and bind their
addresses once. Update __call__ to copy prepared inputs into the existing input
buffers and reuse the existing output buffers instead of calling aligned_tensor
for each invocation, while preserving state-buffer handling and returned
outputs.
examples/onnx_ptq/far3d/quantize.py (1)

76-99: 🎯 Functional Correctness | 🔵 Trivial | 💤 Low value

Defensive gap: empty/duplicate node.name silently truncates the exclusion BFS.

nodes_by_name[node.name] = node overwrites on name collisions, and the BFS's if not name: continue (line 94) drops any node with an empty name from excluded and stops traversing its outputs — so any accuracy-sensitive node reachable only through an unnamed node would be missed. Likely low-probability given this graph's nodes carry meaningful names (OSA4_5, lateral_convs), but worth a defensive guard since a silent miss here degrades accuracy without any error.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@examples/onnx_ptq/far3d/quantize.py` around lines 76 - 99, The
find_encoder_nodes_to_exclude function must preserve BFS traversal through
unnamed and duplicate-name nodes instead of indexing nodes solely by node.name.
Track graph nodes and their outputs using stable node identities or an adjacency
structure that does not overwrite collisions, include unnamed nodes in
traversal, and only use names when adding valid exclusions while continuing
through every reachable node.
examples/onnx_ptq/far3d/Dockerfile (1)

1-81: 🔒 Security & Privacy | 🔵 Trivial | 💤 Low value

Root user (Trivy DS-0002).

No USER directive; container runs as root. This matches other example Dockerfiles in the repo and the README's documented --privileged --gpus=all workflow, so the practical risk is low for a local example build.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@examples/onnx_ptq/far3d/Dockerfile` around lines 1 - 81, Add a non-root
runtime user and switch to it with a USER directive in the Dockerfile, ensuring
the user can access and execute the installed /opt/far3d and /opt/modelopt
contents. Preserve the existing dependency installation steps, which must
continue running as root before the user switch.

Source: Linters/SAST tools

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@examples/onnx_ptq/far3d/quantize.py`:
- Around line 118-133: The nodes passed by quantize_encoder through
nodes_to_exclude must match only the exact excluded node names. Transform the
result of find_encoder_nodes_to_exclude into escaped, fully anchored regex
patterns before supplying it to quantize, preserving the existing exclusion set
without allowing prefix or metacharacter-based matches.

---

Nitpick comments:
In `@examples/onnx_ptq/far3d/Dockerfile`:
- Around line 1-81: Add a non-root runtime user and switch to it with a USER
directive in the Dockerfile, ensuring the user can access and execute the
installed /opt/far3d and /opt/modelopt contents. Preserve the existing
dependency installation steps, which must continue running as root before the
user switch.

In `@examples/onnx_ptq/far3d/evaluate.py`:
- Around line 257-260: Move import_modules_from_strings from mmcv.utils into the
top-level import section of evaluate.py, alongside the existing mmcv import, and
remove the local import inside the cfg.get("custom_imports") conditional.
Preserve the conditional invocation and its existing arguments.
- Around line 59-142: Pre-allocate reusable non-state input and output buffers
in TensorRTRunner.__init__, using the fixed input_shapes, output_shapes, and
tensor_dtypes, and bind their addresses once. Update __call__ to copy prepared
inputs into the existing input buffers and reuse the existing output buffers
instead of calling aligned_tensor for each invocation, while preserving
state-buffer handling and returned outputs.

In `@examples/onnx_ptq/far3d/quantize.py`:
- Around line 76-99: The find_encoder_nodes_to_exclude function must preserve
BFS traversal through unnamed and duplicate-name nodes instead of indexing nodes
solely by node.name. Track graph nodes and their outputs using stable node
identities or an adjacency structure that does not overwrite collisions, include
unnamed nodes in traversal, and only use names when adding valid exclusions
while continuing through every reachable node.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 85ac9fde-ebb2-416e-9794-10fdd8a25573

📥 Commits

Reviewing files that changed from the base of the PR and between d984de3 and a799d6f.

📒 Files selected for processing (10)
  • .pre-commit-config.yaml
  • LICENSE
  • examples/onnx_ptq/README.md
  • examples/onnx_ptq/far3d/Dockerfile
  • examples/onnx_ptq/far3d/README.md
  • examples/onnx_ptq/far3d/evaluate.py
  • examples/onnx_ptq/far3d/far3d_pytorch_25_06.patch
  • examples/onnx_ptq/far3d/prepare_calibration.py
  • examples/onnx_ptq/far3d/prepare_metadata.py
  • examples/onnx_ptq/far3d/quantize.py

Comment thread examples/onnx_ptq/far3d/quantize.py
Signed-off-by: ajrasane <131806219+ajrasane@users.noreply.github.com>
@ajrasane
ajrasane requested a review from a team as a code owner July 27, 2026 16:18
@ajrasane ajrasane self-assigned this Jul 27, 2026

@cjluo-nv cjluo-nv left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bot review (bedrock-claude-opus-4-8) — DM the bot to share feedback.

Re-review of PR #2012 (FAR3D ONNX PTQ + Argoverse 2 eval example, +910/-0, 11 files). The new code remains sound example glue — modelopt.onnx.quantization.quantize args match the current signature, find_encoder_nodes_to_exclude BFS is correct (all OSA4_5 nodes + everything downstream of lateral_convs), and no new subsystem is introduced. No prompt-injection attempts in the untrusted content (CodeRabbit's "Autofix"/"AI Agents" prose is tooling boilerplate, not directives I acted on).

Addressed since the prior review:

  • 💬 CHANGELOG concern resolved — the diff now adds a proper CHANGELOG.rst bullet under the Experimental section for the FAR3D example.

Why nudge (not approve):

  1. Licensing — human sign-off required (cannot auto-approve). Unchanged from the prior review: this PR edits the top-level LICENSE (adds Copyright (c) OpenMMLab. All rights reserved. under Apache-2.0), evaluate.py carries stacked OpenMMLab/"Modified by Zhiqi Li" + NVIDIA SPDX headers (adapted from DL4AGX/megvii Far3D), and vendors far3d_pytorch_25_06.patch modifying third-party FAR3D source. The attribution follows CONTRIBUTING.md's documented third-party format (reference link w/ commit hash → upstream copyright → NVIDIA header, LICENSE updated, file excluded from the license hook) and looks conscientious — but per focus area 6 and CONTRIBUTING's OSRB requirement this needs a human/legal reviewer to sign off.
  2. nodes_to_exclude not anchored (CodeRabbit "major", still open). Confirmed expand_node_names_from_patterns in modelopt/onnx/quantization/graph_utils.py resolves exclusions via re.match(pattern, node.name), so raw names act as unanchored prefix patterns and any regex metacharacter (., +, () in a node name is interpreted specially — a bad metachar could even raise re.error. For this example the over-match direction is conservative (keeps more nodes in high precision), so it's hardening rather than an accuracy bug, but worth escaping+anchoring (re.escape(name) + ^…$).
  3. No tests / no CI path. Consistent with the example-dir convention (GPU/TRT/mmdet3d-only), but the self-contained find_encoder_nodes_to_exclude could get a small unit test if the team wants any coverage. Owner's call.

Minor CodeRabbit nits still open (non-blocking): local mmcv.utils import in evaluate.py should move to top of file (no stated circular/optional/heavy justification, and mmcv is already imported at module top); TRT I/O buffers reallocated per call; Dockerfile runs as root. Recommend owner (ajrasane / onnx codeowners) sign off, primarily on the licensing.

Signed-off-by: ajrasane <131806219+ajrasane@users.noreply.github.com>

@cjluo-nv cjluo-nv left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bot review (bedrock-claude-opus-5) — DM the bot to share feedback.

Re-review of PR #2012 (FAR3D ONNX PTQ + Argoverse 2 eval example, +913/-0, 11 files).

Short answer to "should I approve?" — Not on my recommendation alone. Everything I can check mechanically is now in good shape, but the licensing surface requires human/OSRB sign-off, so I can't auto-approve. If you already have (or can get) legal/OSRB confirmation on the third-party attribution, this is fine to merge from a code perspective.

Resolved since the prior review:

  • 💬 nodes_to_exclude anchoring — addressed in commit 1808c72. Verified in the current file: quantize_encoder now builds rf"^{re.escape(name)}$" patterns, which is exactly right given expand_node_names_from_patterns resolves exclusions with re.match. No more prefix/metacharacter over-match.
  • 💬 CHANGELOG — resolved; an Experimental bullet for the FAR3D example is now in CHANGELOG.rst.

Correctness re-check (no new issues): modelopt.onnx.quantization.quantize kwargs all match the current signature; find_encoder_nodes_to_exclude is a single forward pass over topologically-sorted nodes that correctly captures all OSA4_5 nodes plus everything downstream of lateral_convs (propagation stops at OSA nodes, which are excluded anyway); calibration readers, decoder temporal-state handling, and the --fp16-decoder path are internally consistent with the README. Still pure example glue — no new subsystem, no duplicated in-repo functionality. No prompt-injection attempts in the untrusted content (CodeRabbit's "AI Agents"/Autofix prose is tooling boilerplate; I did not treat it as instructions).

Why nudge rather than approve:

  1. Licensing — human sign-off required. Unchanged: top-level LICENSE gains Copyright (c) OpenMMLab. All rights reserved. under Apache-2.0; evaluate.py carries stacked OpenMMLab / "Modified by Zhiqi Li" + NVIDIA SPDX headers (adapted from DL4AGX test_tensorrt.py, itself from megvii-research/Far3D); far3d_pytorch_25_06.patch vendors a modification to third-party FAR3D source. Attribution follows CONTRIBUTING's documented format (reference link with commit hash → upstream copyright → NVIDIA header, LICENSE updated, file excluded from the license pre-commit hook) and looks conscientious — but this is a legal call, not mine.
  2. No tests / no CI-exercisable path. Consistent with the examples/ convention and the GPU/TRT/mmdet3d-only workflow. find_encoder_nodes_to_exclude remains the one self-contained piece that could get a small unit test — owner's call.

Minor, non-blocking (still open): the from mmcv.utils import import_modules_from_strings inside main() (evaluate.py ~L257) should move to the top of the file — mmcv is already imported at module level and no circular/optional/heavy justification is given. Also unchanged: TRT I/O buffers reallocated per __call__ (hot path across 23.5k frames), and the Dockerfile runs as root (matches other example images).

Signed-off-by: ajrasane <131806219+ajrasane@users.noreply.github.com>
Comment thread examples/onnx_ptq/far3d/README.md Outdated
far3d.decoder.${precision}.engine
```

The evaluator reports the dataset metrics, including mAP. The DL4AGX reference reports 0.230 mAP for its INT8 encoder and mixed FP16/FP32 decoder, compared with 0.232 mAP for FP16 encoder and decoder. It reports neither an INT8 decoder nor FP8 results; validate the selected precision on the target TensorRT version and GPU.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't have a good understanding on this paragraph. How about we just come up with a table showing the scores of the various formats?

Signed-off-by: ajrasane <131806219+ajrasane@users.noreply.github.com>
# See the License for the specific language governing permissions and
# limitations under the License.

FROM nvcr.io/nvidia/pytorch:25.06-py3

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are we using 1 year old docker?

Comment on lines +31 to +80
RUN env -u PIP_CONSTRAINT /opt/far3d/bin/python -m pip install --no-cache-dir \
torch==1.13.1+cu117 \
torchvision==0.14.1+cu117 \
--extra-index-url https://download.pytorch.org/whl/cu117

RUN env -u PIP_CONSTRAINT /opt/far3d/bin/python -m pip install --no-cache-dir \
"numpy<1.24" \
opencv-python==4.5.5.64 \
yapf==0.32.0 \
mmcv-full==1.7.0 \
-f https://download.openmmlab.com/mmcv/dist/cu117/torch1.13.0/index.html

RUN env -u PIP_CONSTRAINT /opt/far3d/bin/python -m pip install --no-cache-dir \
av2==0.2.1 \
einops \
"ipython<9" \
kornia==0.6.12 \
mmdet==2.28.2 \
mmsegmentation==0.30.0 \
onnx \
onnx-graphsurgeon==0.6.1 \
onnxruntime \
onnxsim \
refile \
tensorrt-cu12-bindings==10.11.0.33 \
--extra-index-url https://pypi.nvidia.com && \
mkdir -p /opt/far3d/lib/python3.8/site-packages/tensorrt && \
cp /opt/far3d/lib/python3.8/site-packages/tensorrt_bindings/__init__.py \
/opt/far3d/lib/python3.8/site-packages/tensorrt/__init__.py && \
cp /opt/far3d/lib/python3.8/site-packages/tensorrt_bindings/tensorrt.so \
/opt/far3d/lib/python3.8/site-packages/tensorrt/tensorrt.so

RUN env -u PIP_CONSTRAINT /opt/far3d/bin/python -m pip install --no-cache-dir \
"setuptools<81" && \
env -u PIP_CONSTRAINT /opt/far3d/bin/python -m pip install --no-cache-dir \
--no-build-isolation \
mmdet3d==1.0.0rc6

COPY . /opt/modelopt
RUN env -u PIP_CONSTRAINT python -m pip install --no-cache-dir \
lief \
ml_dtypes \
omegaconf \
"onnx-graphsurgeon>=0.6.1" \
"onnx~=1.21.0" \
"onnxruntime-gpu~=1.24.2" \
onnxscript \
"onnxslim>=0.1.76" \
"setuptools>=80" && \
env -u PIP_CONSTRAINT python -m pip install --no-cache-dir --no-deps -e /opt/modelopt

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you move all requirements.txt and install from it instead of mentioning in dockerfile so nspect scanning can detect these dependencies

Comment on lines +97 to +98
parser.add_argument("calibration_dir", help="Directory created by prepare_calibration.py")
parser.add_argument("--quantization-mode", choices=("int8", "fp8"), default="int8")

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we use same approach either parser.add_argument("xyz") or parser.add_argument("--xyz")?

Also can we use either --arg-name or --arg_name for consistency

Comment on lines +69 to +70
COPY . /opt/modelopt
RUN env -u PIP_CONSTRAINT python -m pip install --no-cache-dir \

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we instead do pip install -e .[onnx] here?

--no-build-isolation \
mmdet3d==1.0.0rc6

COPY . /opt/modelopt

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we use /opt/Model-Optimizer naming? Thats the convention we use in other places also (e.g. Megatron-Bridge)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants