arXiv:2610.38095 · v1
Nemotron 3.5 Super and Lightning: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Models for Agentic AI
NVIDIA:Aastha JhunjhunwalaAayushi Ashokkumar SrivastavaAbhibha GuptaAbhinav GargAbhinav KhattarAchsah WildishAdam RajferAdi RenduchintalaAdil AsifAdriana Flores MirandaAileen ZamanAishwarya ChhabraAkanksha ShuklaAkhiad BercovichAlejandra RicoAleksa GordicAleksander FicekAlex GronskiyAlex SteinerAlexander BukharinAlexandre MilesiAli Roshan GhiasAli ShehperAli TaghibakhshiAlice GattiAlisa LiuAlok KumarAlok TripathyAmala Sanjay DeshmukhAman MiryalaAman ShanbhagAmeya Sunil MahabaleshwarkarAmeya VelingkerAmir KleinAmir SamaniAmit ZukerAmnon GeifmanAnahita BhiwandiwallaAnanth SubramaniamAndre ManoelAndrea SantilliAndrew TaoAndrii SkliarAnil ThomasAnish MahishiAnna ShorsAnna WarnoAnuj DoshiAnusha KamathAnwith KiranAo TangArham MehtaArnab DharaArnav KomaragiriArnav KunduArti JainArun RaoArun VenkatesanArushi GoelAsha AnooshehAshish VaswaniAshwath AithalAsif AhamedAsma Kuriparambil ThekkumpateAswin VisvaAtefeh SohrabizadehAvinash VemAyush DattaguptaBenedikt SchiffererBenjamin ChislettBernard NguyenBesmira NushiBilal KartalBita Darvish RouhaniBobby ChenBoris GinsburgBoxin WangBrandon NorickBranislav KisacaninBrendan JohnsonBrian DangBrian YuBryan CatanzaroByung-Kwan LeeCarlo del MundoChankyu LeeChantal HwangCharles WangCharlie TruongChen CuiCheng WangCheng-Ping HsiehChenhan YuChenjie LuoChintan PatelChris AlexiukChris JohnsonChris WingChristian MunleyCoco LiCollin McCarthyCynthia GaoDan SuDan ZhaoDane CorneilDanial Mohseni TaheriDaniel AfrimiDaniel FatadeDaniel LoDaniel MachlabDaniel SerebrenikDaniele MolinariDaniil SorokinDaria GitmanDaria LevyDavid MosallanezhadDavid YuDeepak NarayananDev ChuriwalaDevin O'KellyDhruv NathawaniDi Allan WuDima RekeshDina YaredDipendra MisraDivyanshu KakwaniDmitry BelenkoDominic MannoDongyang FanDor TsurDuncan RiachDustin VanSteeDylan LuEdgar MinasyanEhsan Hosseini-AslEileen LongElena LantzEllie EvansElliott NingElvis ChenEmanuel ScoullosEric ChungEric HarperEric Pham-HungEric TramelEric YangErik ArakelyanEvelina BakhturinaEwa DobrowolskaFadi SaadyFarzan MemarianFay WangFei JiaFelipe SoaresFelipe Vieira FrujeriFeng ChenFengguang LinFerenc GalkoFrankie SiinoFrederic LafranceFuxiao LiuGantavya BhattGarvit KulshreshthaGeorge ArmstrongGiulio LovisottoGomathy Venkata KrishnanGosia LazukaGrace LamGreg HeinrichGreg PauloskiGrzegorz ChlebusGuilin LiuGuo ChenGuoming ZhangGuyue HuangHaifeng QianHanrong YeHao GaoHao ZhangHaoqin TuHaran Kumar Shiv KumarHaribhau HudHarris NoverHarrison Saturley-HallHarshil KotamreddyHemil DesaiHenry EstelaHexin WangHimanshu SinghHui WangHung-Yueh ChiangIgor GitmanIlia KarmanovIlya LoshchilovIngo GuehringItay NeemanIvan MoshkovJacky KwokJakub SlowikowskiJames ShenJane Polak ScowcroftJared CasperJatin MitraJeff WilletteJenna DiamondJenny ChenJesse OliverJessica MedinaJiacheng XuJialin SongJian HuJian TuJian ZhangJiantao JiaoJiaqi ZengJie-Fang ZhangJimmy ZhangJin XuJingyu XinJinhang ChoiJoey ConwayJoey GumanJohn KamaluJohn LiuJohn RobertsJohnny GrecoJonah AlbenJonas YangJonathan CohenJonathan LiJonathan RaimanJoseph JenningsJuhi MittalJulia GusakJulien Veron VialardJune YangJunkeun YiJupinder ParmarKa Shun ShumKaiwen ShiKajal JainKanishk SinglaKaran SapraKari BriskiKateryna ChumachenkoKatherine HuangKatie LyonsKeith WyssKeshav SanthanamKeval MorabiaKevin ShihKezhi KongKhanh NguyenKhushi BhardwajKiran PraveenKonstantinos KrommydasKrishna PuvvadaKrzysztof PawelecKushal ShahLi DingLily LeeLinda LiuLinglin JingLuis VegaMaarten Van SegbroeckMahan FathiMahdi NazemiMaka DongMakesh Narsimhan SreedharMaor AshkenaziMarc CuevasMarcin ChochowskiMarco PavoneMarkus KlieglMarta Stepniewska-DziubinskaMartina Gonzalez VilasMartyna PatelkaMatan RusanovskyMatthieu LeMatty MachczynskiMatvei NovikovMauricio FerratoMeenakshi SharmaMehrzad SamadiMelissa CorpuzMelissa SaelzerMeng XinMengru WangMengxi WuMeriem BoubdirMichael BooneMichael CallahanMichael EladMichael LightstoneMichal BienMichal ZawalskiMiguel MartinezMikail KhonaMike ChrzanowskiMike RanzingerMike WojcikiewiczMin-Hung ChenMing KangMingjie LiuMingyuan MaMisha NekorystnovMJ MikulskiMohammad Qazim BhatMohammad ShoeybiMorgan PurcellMostofa PatwaryNabin MulepatiNancy AgarwalNarimane HennouniNathan SalaNave AssafNayeon LeeNeal VaidyaNegar HabibiNetanel HaberNick GuptaNick ReamaroonNickson QuakNidhi BhatiaNik SpirinNiket AgarwalNikolai LudwigNima TajbakhshNithin SunnyOleksii HrinchukOleksii KuchaievOlivier DelalleauOmer Ullman ArgovPablo RibaltaPadmavathy SubramanianParth MannanPavlo MolchanovPawel MorkiszPeter JinPetr LapukhovPetro Junior MilanPhilip MonkPinky XuPiotr JanuszewskiPiotr LaszkiewiczPo-Han HuangPraateek MahajanPradeep ThalastaPrakash GurumurthyPranav Prashant ThombrePrasoon VarshneyPratyusha MaitiPreeti ChadhaPritam BiswasPritam GundechaPrzemek StrzelczykPuhui MengPulkit KumarRabeeh Karimi MahabadiRachel ObermanRajpreet ThethyRamani AkkatiRameshwar ShivbhaktaRan ZilbersteinRandy GelhausenRasoul ShafipourRaviraj JoshiRay MacalisangRayen TianReka KovacsRenjie PiResha ShettyRia CRima ShahbazyanRishabh GargRita Fernandes NevesRitika BorkarRitu GalaRiya TalwarRiyad IslamRobert ClarkRobert KirbyRoger WaleffeRohan JoshiRohan PillaiRohit JenaRohit WatveRoi GarnettRoi KorenRongguang YeRussell J. HewettRyan PrengerRyan StewartRyota EgashiraSadegh MahdaviSai Nishanth PadalaSalika DaveSandhya Rani NarravulaSandip BhaskarSangkug LymSanjeev SatheeshSarah AmirslaniSaran Vikas MurariSaurabh MishraSaurabh SrivastavaSaurav MuralidharanSean NarenthiranSean WangSeokju ChoSeong Hee LeeSeonjin NaSeonmyeong BakSeph MardSerge PanevSergei KolchenkoSergio PerezSeth HennemanSeth PoulosShahar MorShantanu AcharyaShaokun ZhangShaona GhoshSharath Turuvekere SreenivasShaun KotekShaya GharghabiSheng-Chieh LinShengliang XuShi ChenShihao WangShilpa AmireddyShiqing FanShiyang ChenShrimai PrabhumoyeShriram VenugopalShuang YuShubham GandhiShubham PachoriShubham ToshniwalShuoyang DingShwetha KrishnamurthySimeng SunSivakumar Arayandi ThottakaraSomshubra MajumdarSoumye SinghalSridurga KrithivasanSriharsha NivertyStephen GeSugam Dipak DevareSukrit RaoSumeet Kumar BaruaSungsoo HaSurya NistalaSuseella PanguluriSweta PriyadarshiSyeda Nahida AkterTalor AbramovichTanay VarshneyTandeep Singh ChandehokeTeo EneTerry KongThanh DoThinh LeTian ZhangTijmen BlankevoortTim RomanskiTiyasa MitraTom BaloughTom HuangTomasz GrzegorzekTomer AsidaTomer Bar NatanTraian RebedeaTrenton StarkeyTugrul KonukTuomas RintamakiTwinkle VashishthTyler KniessTyler PoonUdi KarpasUshnish DeUtkarsh VaidyaVahid NorooziVanshil Atul ShahVenkat SrinivasanVenkata KarriVenmugil ElangoVibhor AgrawalVibhu JawaVictor CuiVictoria EdmondsVijay KorthikantiVikas MehtaVikranth JeyakumarVinay RaoVineeth Kalluru SrinivasVitaly KurinVitaly LavrukhinWahidur RahmanWanli JiangWasi Uddin AhmadWataru IshiharaWei DuWei PingWei-Ming ChenWeiheng ChaiWeijia ChenWendy QuanWenliang DaiWenyan CongWilliam ZhangWojciech PrazuchXiaowei RenXin YuXiwen YuYan WangYang ChenYang YuYangyi ChenYaniv GalronYash JainYashaswi KarnatiYe YuYejin ChoiYemi IbrahimYev MeyerYi DongYi-Fu WuYian ZhangYifan ZhangYing LinYiyang ZhouYoav MironYogendra PandeyYonatan GeifmanYonggan FuYoshi SuharaYoungeun KwonYu-Cheng ChouYu-Chiang Frank WangYuanguo KuangYuekai ZhangYugi GuvvalaYuki HuangYunheng ZouYunsheng LiuZhenming WangZhiding YuZhiqi LiZhiyu ChengZhongbo ZhuZhuolin YangZichao YueZihan LiuZijie YanZirui LiuZoe FaulZuhair Ahmed
首次提交 2026-10-28 08:28
版本更新 2026-10-28 08:28
雷达观测 2026-10-30 10:23
摘要
We introduce Nemotron 3.5 Super and Nemotron 3.5 Lightning, open and efficient Mixture-of-Experts Hybrid Mamba-Transformer models designed for reasoning, long-context understanding, and agentic workloads. Super is a 120B-parameter, 12B-active multimodal model, while Lightning is a smaller 30B-parameter, 3B-active text model; both designed for efficient deployment. Both models combine large-scale distillation, long-context training, and native speculative decoding, while Super further incorporates Indic continued pretraining and agentic software-engineering mid-training. We develop a comprehensive post-training pipeline spanning supervised fine-tuning and reinforcement learning (RLVR and RLHF), complemented in Super by specialized teacher training and multi- teacher on-policy distillation (MOPD). This transforms Super from a text-only base model into a multimodal agent capable of reasoning over text, images, video, and interactive tool environments. We further explore test-time scaling through parallel agent execution and self-verification. Across training and deployment, we jointly optimize capability and efficiency. Indic continued pretraining improves Indian-language benchmarks by up to 9.82 percentage points, while agentic mid-training strengthens software-engineering capabilities and robustness across agent harnesses. The post-trained Super model achieves 51.78% on SWE-bench Pro, surpassing Nemotron 3 Ultra despite using approximately 4.6x fewer active parameters. Post-training MTP boosting improves speculative- decoding acceptance length by up to 21.8% under probabilistic draft sampling, while NVFP4 quantization achieves 99.62% median accuracy recovery relative to BF16. Inference optimizations enable up to 4,856 output tokens/s/GPU on GB200 while sustaining at least 50 tokens/s/user.