BurstGPT
A real-world view of LLM serving: 10.31 million workload traces from Azure OpenAI services, collected over 213 days.
I work as a distributed systems expert at ModelBest. I architect and build AI infrastructure, delivering efficient, reliable, and observable systems. I also work on systems that can improve themselves over time.
More background
Previously, I was a researcher at HKRC, focusing on reliability in hyperscale systems, self-evolving AI infrastructure, and software design for coding agents. I received my Ph.D. in Computer Science from HKBU, supervised by Prof. Xiaowen Chu and Dr. Amelie Chi Zhou. I was also a visiting student at NUS, advised by Prof. Bingsheng He. I received my B.Eng. in Electrical Engineering from HUST.
Open source projects
AFD & Elastic AFD
Disaggregating attention and FFN for efficient MoE inference, with elastic scaling through traffic spikes and node failures.
Elastic & Fault-Tolerant EP
Process-level recovery for distributed MoE inference, combining fault detection, device isolation, and elastic expert parallelism.
Recent publications
Zellige: Moldable Sequence Placement for Mixed Image-Video DiT Training
KernelFlume: Elastic Core-Attention Scaling for Agentic Long-Context Decoding
BurstGPT: A Real-world Workload Dataset to Optimize LLM Serving Systems
Fault-Tolerant Hybrid-Parallel Training at Scale with Reliable and Efficient In-memory Checkpointing
Xema: Efficient Diffusion Serving through Fine-Grained Memory Management and Auto-Configuration
RcLLM: Accelerating Generative Recommendation via Beyond-Prefix KV Caching
ElasWave: An Elastic-Native System for Scalable Hybrid-Parallel Training
Full list on Google Scholar
1,258 citations · August 2026
Updates
- Zellige is available on arXiv.
- Xema is available on arXiv.
- KernelFlume is available on arXiv.
- RcLLM was accepted at ICDCS 2026. Congrats to Zhan!
- Our paper on gradient aggregation errors in fault-tolerant training was accepted at ICML 2026. Congrats to Zhenheng!
- DreamDDP was accepted at MLSys 2026. Congrats to Zhenheng!
- ExpertFlow was accepted at DAC 2026. Congrats to Xin!
- Our paper on disaggregated MoE inference is available on arXiv.
- ElasWave is available on arXiv.
- Our paper on serving reasoning models was accepted at ICLR 2026. Congrats to Qi!
- I proposed ATTN-FFN Disaggregation and Elastic AFD for MoE models in vLLM for efficient DeepSeek-v3/R1 serving. Now advancing as an experimental feature.
- BurstGPT was accepted at KDD 2025.
- Our paper on in-memory checkpointing for fault-tolerant training is available on arXiv.
- Our work on bandwidth-aware compression for federated learning was accepted at ICPP 2024. Congrats to Zichen!