Fetch from nvidia Megatron-LM by RaymondLi0 · Pull Request #5 · ElementAI/Megatron-LM

RaymondLi0 · 2022-08-03T20:32:25Z

No description provided.

Signed-off-by: oliver könig <okoenig@nvidia.com>

Signed-off-by: Charlie Truong <chtruong@nvidia.com>

Signed-off-by: oliver könig <okoenig@nvidia.com>

Co-authored-by: root <root@gpu-h100-0348.cm.cluster> Co-authored-by: root <root@gpu-h100-0193.cm.cluster> Co-authored-by: root <root@gpu-h100-0082.cm.cluster> Co-authored-by: root <root@gpu-h100-0495.cm.cluster> Co-authored-by: William Dykas <wdykas@cw-pdx-cs-001-vscode-02.cm.cluster> Co-authored-by: root <root@gpu-h100-0213.cm.cluster> Co-authored-by: root <root@gpu-h100-0435.cm.cluster> Co-authored-by: root <root@gpu-h100-0188.cm.cluster> Co-authored-by: root <root@gpu-h100-0032.cm.cluster> Co-authored-by: root <root@gpu-h100-0023.cm.cluster> Co-authored-by: root <root@gpu-h100-0368.cm.cluster> Co-authored-by: root <root@gpu-h100-0203.cm.cluster> Co-authored-by: root <root@gpu-h100-0229.cm.cluster> Co-authored-by: root <root@gpu-h100-0123.cm.cluster> Co-authored-by: root <root@gpu-h100-0217.cm.cluster> Co-authored-by: root <root@gpu-h100-0496.cm.cluster> Co-authored-by: root <root@gpu-h100-0261.cm.cluster>

…ernorm. (#2434) Co-authored-by: Yuzhong Wang <yuzhongw@nvidia.com>

Signed-off-by: Hongbin Liu <hongbinl@nvidia.com>

Signed-off-by: Youngeun Kwon <youngeunk@nvidia.com> Co-authored-by: Youngeun Kwon <youngeunk@nvidia.com>

Signed-off-by: Keshav Santhanam <ksanthanam@nvidia.com>

This reverts commit ffbc43f.

Signed-off-by: oliver könig <okoenig@nvidia.com>

Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com>

Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Xin Yao <xiny@nvidia.com>

Co-authored-by: Rabeeh Mahabadi <rkarimimahab@nb-hel-cs-001-vscode-02.cm.cluster> Co-authored-by: Sanjeev Satheesh <sasatheesh@nvidia.com> Co-authored-by: Deepak Narayanan <dnarayanan@nvidia.com>

Signed-off-by: Santosh Bhavani <santosh.bhavani@live.com> Co-authored-by: Xin Yao <xiny@nvidia.com>

…offload` (#2874)

Co-authored-by: Maanu Grover <109391026+maanug-nv@users.noreply.github.com>

Signed-off-by: Deepak Narayanan <dnarayanan@nvidia.com>

Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com>

…3194)" This reverts commit e836e62.

This reverts commit 300d1b6.

Co-authored-by: Antoni-Joan Solergibert <asolergibert@nvidia.com>

Signed-off-by: Hollow Man <hollowman@opensuse.org> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Xin Yao <xiny@nvidia.com>

Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com>

Signed-off-by: Robin Zhang <robinz@nvidia.com>

…passes (#3525)

Signed-off-by: oliver könig <okoenig@nvidia.com>

Signed-off-by: Maanu Grover <maanug@nvidia.com>

Co-authored-by: Xin Yao <xiny@nvidia.com>

Signed-off-by: Guyue Huang <guyueh@oci-hsg-cs-001-vscode-01.cm.cluster> Co-authored-by: Guyue Huang <guyueh@oci-hsg-cs-001-vscode-01.cm.cluster> Co-authored-by: Xin Yao <xiny@nvidia.com>

Signed-off-by: oliver könig <okoenig@nvidia.com>

Signed-off-by: Charlie Truong <chtruong@nvidia.com>

…3591)

Co-authored-by: Xin Yao <xiny@nvidia.com>

…3300) Co-authored-by: Xin Yao <xiny@nvidia.com>

Signed-off-by: nvidia <nvidia@TRY-64956-gpu01.nvidialaunchpad.com>

#3630)

Signed-off-by: Faradawn Yang <73060648+faradawn@users.noreply.github.com>

… Manager patch, docs (#3507) Signed-off-by: Faradawn Yang <73060648+faradawn@users.noreply.github.com>

Signed-off-by: Keshav Santhanam <ksanthanam@nvidia.com>

ko3n1g and others added 30 commits January 30, 2026 09:46

ci: Onboard more GB200 tests (#3145)

6de6362

Signed-off-by: oliver könig <okoenig@nvidia.com>

ci(hotfix): Alert for GB200 (#3168)

de15117

Signed-off-by: oliver könig <okoenig@nvidia.com>

Fix SFTDataset truncation bug (#3158)

7952d7e

Vitalyk/multiturn v2 (#3167)

b9ee19e

ci: Disable the api check for now (#3157)

b168849

Signed-off-by: Charlie Truong <chtruong@nvidia.com>

ci: Add DSv3 proxy (#3169)

a205538

Signed-off-by: oliver könig <okoenig@nvidia.com>

Update copy-pr-bot.yaml [skip ci]

fdc04f6

[Community][Main] fix(moe): Fix theoretical memory calculation of lay…

9ad5906

…ernorm. (#2434) Co-authored-by: Yuzhong Wang <yuzhongw@nvidia.com>

fix: Set --refit-method default to gloo (#3172)

5415e1d

[fix] Bug fix for offloading in evaluate() (#3043)

a976754

Signed-off-by: Hongbin Liu <hongbinl@nvidia.com>

Update copy-pr-bot.yaml [skip ci]

991c38f

cp: Fix: nccl-ub in ddp path (3181) into main (#3182)

5d0a7fd

Signed-off-by: Youngeun Kwon <youngeunk@nvidia.com> Co-authored-by: Youngeun Kwon <youngeunk@nvidia.com>

Miscellaneous inference cleanup (#2955)

ffbc43f

Signed-off-by: Keshav Santhanam <ksanthanam@nvidia.com>

Revert "Miscellaneous inference cleanup (#2955)"

0fe3232

This reverts commit ffbc43f.

ci: Fix DSv3 (#3188)

69a5c63

Signed-off-by: oliver könig <okoenig@nvidia.com>

Fix missing argument in MoELayer.forward() (#3133)

2fadde8

Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com>

Fix H2D stream synchronization in optimizer offload (#3140)

ae67076

Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Xin Yao <xiny@nvidia.com>

Add MTP support for hybrid models (#2363)

300d1b6

Co-authored-by: Rabeeh Mahabadi <rkarimimahab@nb-hel-cs-001-vscode-02.cm.cluster> Co-authored-by: Sanjeev Satheesh <sasatheesh@nvidia.com> Co-authored-by: Deepak Narayanan <dnarayanan@nvidia.com>

docs: improve Megatron-LM and Megatron Core descriptions (#3115)

dceb1fb

Signed-off-by: Santosh Bhavani <santosh.bhavani@live.com> Co-authored-by: Xin Yao <xiny@nvidia.com>

Handle step key correctly in checkpoint save with `--optimizer-cpu-…

f4502eb

…offload` (#2874)

mRoPE for MTP (#3114)

70719cd

Co-authored-by: Maanu Grover <109391026+maanug-nv@users.noreply.github.com>

Fix two minor bugs in MTP implementation for hybrid models (#3194)

e836e62

Signed-off-by: Deepak Narayanan <dnarayanan@nvidia.com>

Update README.md (#2111)

1362e4a

Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com>

Revert "Fix two minor bugs in MTP implementation for hybrid models (#…

31d0c87

…3194)" This reverts commit e836e62.

Revert "Add MTP support for hybrid models (#2363)"

a0cc8ca

This reverts commit 300d1b6.

Fix bug in SFTDataset (#3185)

50546da

Co-authored-by: Antoni-Joan Solergibert <asolergibert@nvidia.com>

Fix several syntax error (#3004)

dff4189

Signed-off-by: Hollow Man <hollowman@opensuse.org> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Xin Yao <xiny@nvidia.com>

Fix for RL Test (#3148)

c4bea0a

Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com>

Fix latent moe flops and backward_dw (#2977)

a4008d0

Signed-off-by: Robin Zhang <robinz@nvidia.com>

yobibyte and others added 30 commits February 25, 2026 17:50

Fix default cuda graph persist arg. Add persist to rl common.sh. (#3584)

b359981

Optimize away add request overheads in dummy ep cuda-graphed forward …

60a25aa

…passes (#3525)

ci: Test docs build (#3583)

e7d80bd

Signed-off-by: oliver könig <okoenig@nvidia.com>

ci: Fix docs build for release (#3597)

80ef2ae

Signed-off-by: oliver könig <okoenig@nvidia.com>

ci: Remove secrets (#3598)

2561a59

Signed-off-by: oliver könig <okoenig@nvidia.com>

ci: Define secrets (#3599)

19444b9

Signed-off-by: oliver könig <okoenig@nvidia.com>

ci: gh-release-from-tag (#3600)

a08ad25

Signed-off-by: oliver könig <okoenig@nvidia.com>

Ko3n1g/ci/remove twine username (#3601)

ac449c4

Signed-off-by: oliver könig <okoenig@nvidia.com>

Add training code to MCore wheel (#3573)

9088d4f

Signed-off-by: Maanu Grover <maanug@nvidia.com>

FP8 attention knob for nvFP4 recipe (#3363)

dd4c0b7

Co-authored-by: Xin Yao <xiny@nvidia.com>

Fix error with --load-main-params-from-ckpt (#3569)

c2782fd

Signed-off-by: Guyue Huang <guyueh@oci-hsg-cs-001-vscode-01.cm.cluster> Co-authored-by: Guyue Huang <guyueh@oci-hsg-cs-001-vscode-01.cm.cluster> Co-authored-by: Xin Yao <xiny@nvidia.com>

ci: Create comment (#3610)

c921891

Signed-off-by: oliver könig <okoenig@nvidia.com>

ci: Skip cleanup-taint-node jobs during deployments (#3612)

9100119

Signed-off-by: oliver könig <okoenig@nvidia.com>

ci: No comment for release workflow (#3615)

6161f7a

Signed-off-by: oliver könig <okoenig@nvidia.com>

ci: Re-add release tag prefix (#3619)

36d8a9d

Signed-off-by: oliver könig <okoenig@nvidia.com>

docs: Fix version picker urls (#3621)

1b1f5c4

Signed-off-by: Charlie Truong <chtruong@nvidia.com>

ci: Increase changelog generation max PRs fetched (#3620)

027e0f3

Signed-off-by: Charlie Truong <chtruong@nvidia.com>

Add debug info to an assert. (#3588)

18c94d2

fix: async_utils: explicit GC in persistent checkpoint worker loop (#…

e1a9ac9

…3591)

Fix: Perform sigmoid calculation in fp32 for aux loss stability (#2765)

5f668c1

Co-authored-by: Xin Yao <xiny@nvidia.com>

fix: forward use_te_activation_func flag in non-MoE GPT layer spec (#…

d3c10df

…3300) Co-authored-by: Xin Yao <xiny@nvidia.com>

Multimodal: Limit transformer version in Dockerfile (#3448)

7e3f670

Signed-off-by: nvidia <nvidia@TRY-64956-gpu01.nvidialaunchpad.com>

Track and plot per-token off-policy in RL (#3515)

6c0b9c6

Revert "Add single-process checkpoint save to avoid forked multiproce… (

a7c207f

#3630)

Multimodal: fix VQA dataset selection (#3464)

6503bf8

Signed-off-by: Faradawn Yang <73060648+faradawn@users.noreply.github.com>

Multimodal: Fix multimodal training example - tokenizer, Triton Cache…

2f549e5

… Manager patch, docs (#3507) Signed-off-by: Faradawn Yang <73060648+faradawn@users.noreply.github.com>

Support TP > GQA for inference (#3627)

afbce84

Signed-off-by: Keshav Santhanam <ksanthanam@nvidia.com>

μP: Maximal Update Parameterization (#3058)

310082a

Add flexible virtual pipeline parallel (fVPP) to hybrid model (#3377)

7418b1b

Explicitly close and join Pool in preprocess_data.py (#3592)

53c5973

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fetch from nvidia Megatron-LM#5

Fetch from nvidia Megatron-LM#5
RaymondLi0 wants to merge 6725 commits intoElementAI:load-iterfrom
NVIDIA:main

RaymondLi0 commented Aug 3, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

20 participants

Conversation

RaymondLi0 commented Aug 3, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

20 participants