Sungmin Yun
I am a Senior Deep Learning Performance Architect at NVIDIA, focusing on AI acceleration, performance analysis, and optimization. My work aims to maximize the performance and efficiency of AI systems through hardware–software co-design.
My experience spans accelerator design, hardware profiling, performance modeling, and kernel programming for modern AI architectures, including MoE, MLA, RAG, and diffusion. My research has focused on efficient LLM serving, Multi-LoRA inference, and RAG pipelines. I also developed LLMSimulator, an open-source end-to-end LLM performance simulator.
Previously, I was a postdoctoral researcher at Seoul National University (SNU). I received my Ph.D. in Artificial Intelligence from SNU, under the supervision of Prof. Jung Ho Ahn in the SCALE Lab. I received my B.S. in Integrated Information Technology from Yonsei University in 2020.
Experience
Deep learning performance analysis. Analyze the performance of deep learning applications, focusing on AI acceleration and performance optimization.
LLM serving systems. Developed system-level guidelines and optimizations for LLM serving, including Multi-head Latent Attention (MLA) and Mixture-of-Experts (MoE) architectures.
Multi-LoRA inference at scale. Led the end-to-end design and implementation of an accelerator for Multi-LoRA inference.
RAG pipeline acceleration. Co-designed search algorithms and hardware for large-scale retrieval-augmented generation pipelines.
HBM-PIM controller design. Designed and implemented a hardware controller in RTL for general matrix–vector multiplication (GEMV) using HBM-PIM.
LLM accelerator architecture. Analyzed the architectural limitations of HBM-PIM and proposed Duplex, an LLM accelerator integrating custom HBMs with xPUs. This work was later published at MICRO 2024.
Education
Publications
-
Clone: A Collaborative Multi-device System for Retrieval-Augmented Generation over CXL
-
Rethinking LLM Inference Bottlenecks: Insights from Latent Attention and Mixture-of-Experts
-
SSD Offloading for LLM Mixture-of-Experts Weights Considered Harmful in Energy Efficiency
-
COSMOS: A CXL-Based Full In-Memory System for Approximate Nearest Neighbor Search
-
Anaheim: Architecture and Algorithms for Processing Fully Homomorphic Encryption in Memory
-
Duplex: A Device for Large Language Models with Mixture of Experts, Grouped Query Attention, and Continuous Batching
-
CLAY: CXL-based Scalable NDP Architecture Accelerating Embedding Layers
-
GraNDe: Efficient Near-Data Processing Architecture for Graph Neural Networks
-
GraNDe: Near-Data Processing Architecture with Adaptive Matrix Mapping for Graph Convolutional Networks
-
TRiM: Enhancing Processor-Memory Interfaces with Scalable Tensor Reduction in Memory
Awards
Invited Presentations
Academic Service
Program Committee
- ISCA 2026 — Lightweight Program Committee Member
Journal Review
- IEEE Transactions on Computers
- IEEE Transactions on Circuits and Systems for Artificial Intelligence
- IEEE Computer Architecture Letters
- Future Generation Computer Systems
- Journal of Supercomputing