I am a Research Sceintist at xAI, previously working at OpenAI. I'm recognized as pretraining lead of Grok2-mini, a core contributor to GPT-4o, and inventor of GPT4-Turbo long-context algorithm, and a primary contributor to DALL-E 3, and the first contributor to OpenAI Embedding. My research interest lies in machine learning, optimization, large language models, numerical methods and theories.

Projects

       Grok2-mini: Pretraining lead
       Grok2: core contributor
       GPT-4o: core contributor
       GPT4-Turbo: core contributor, sole inventor of long-context algorithm
       DALL · E3: primary contributor
       OpenAI Embedding model: first contributor
       GPT-4: co-author

Sample Packages

1. AdaBelief optimizer implemented in [PyTorch], [Tensorflow-Addons], [Google Flax], [Deepmind Optax].
2. [TorchDiffEqPack] for accurate and memory-efficient ODE solvers for deep learning.

Publications

Improving Image Generation with Better Captions
GPT-4 Technical Report
Surrogate Gap Minimization improves Sharpness-Aware Training
ICLR 2022
Juntang Zhuang, Boqing Gong, Liangzhe Yuan, Yin Cui, Hartwig Adam, Nicha Dvornek, Sekhar Tatikonda, James S. Duncan, Ting Liu
Momentum Centering and Asynchronous Update for Adaptive Gradient Methods
NeurIPS 2021
Juntang Zhuang, Yifan Ding, Tommy Tang, Nicha Dvornek, Sekhar Tatikonda, James S. Duncan
MALI: A memory efficient and reverse accurate integrator for Neural ODEs
ICLR 2021
Juntang Zhuang, Nicha Dvornek, Sekhar Tatikonda, James S. Duncan
Multiple-shooting adjoint method for whole-brain dynamic causal modeling
IPMI 2021 (Oral presentation)
Juntang Zhuang, Nicha Dvornek, Sekhar Tatikonda, Xenophon Papademetris, Pamela Ventola, James S. Duncan
AdaBelief Optimizer: Adapting Stepsizes by the Belief in Observed Gradients
NeurIPS 2020 (Spotlight, Top 5%)
Juntang Zhuang, Tommy Tang, Yifan Ding, Sekhar Tatikonda, Nicha Dvornek, Xenophon Papademetris, James S. Duncan
Adaptive Checkpoint Adjoint Method for Gradient Estimation in Neural ODE
ICML 2020
Juntang Zhuang, Xiaoxiao Li, , Nicha Dvornek, Sekhar Tatikonda, Xenophon Papademetris, James S. Duncan
Decision Explanation and Feature Importance for Invertible Networks
ICCV 2019, XAIC (Oral presentation)
Juntang Zhuang, Nicha C. Dvornek, Xiaoxiao Li, Junlin Yang, James S. Duncan
ShelfNet for fast semantic segmentation
ICCV 2019, CVRSUAD
Juntang Zhuang, Junlin Yang, Lin Gu, Nicha C. Dvornek.
Invertible Network for Classification and Biomarker Selection for ASD
MICCAI 2019
Juntang Zhuang, Nicha C. Dvornek, Xiaoxiao Li, Junlin Yang, James S. Duncan.
Domain-Agnostic Learning with Anatomy-Consistent Embedding for Cross-Modality Liver Segmentation
ICCV 2019, VRMI
Junlin Yang, Nicha C. Dvornek, Juntang Zhuang, Julius Chapiro, Mingde Lin, James S. Duncan.
LadderNet: Multi-path networks based on U-Net for medical image segmentation
arXiv, 2018
Juntang Zhuang
Prediction of treatment outcome for autism from structure of the brain based on sure independence screening
ISBI 2019
Juntang Zhuang, Nicha C. Dvornek, Qingyu Zhao, Xiaoxiao Li, Pamela Ventola, James S. Duncan.
Prediction of Pivotal response treatment outcome with task fMRI using random forest and variable selection
ISBI 2018
Juntang Zhuang, Nicha C. Dvornek, Xiaoxiao Li, Daniel Yang, Pamela Ventola, James S. Duncan
Prediction of Severity and Treatment Outcome for ASD from fMRI
MICCAI 2018, PRIME Workshop
Juntang Zhuang, Nicha C. Dvornek, Xiaoxiao Li, Pamela Ventola, James S. Duncan.
Brain biomarker interpretation in ASD using deep learning and fMRI
MICCAI 2018
Xiaoxiao Li, Nicha C. Dvornek, Juntang Zhuang, Pamela Ventola, James S. Duncan.
2-channel convolutional 3D deep neural network (2CC3D) for fMRI analysis: ASD classification and feature learning
ISBI 2018
Xiaoxiao Li, Nicha C. Dvornek, Juntang Zhuang, Pamela Ventola, James S. Duncan.