Beyond Zero-Skipping: Redundancy-Aware Architectures and Systems for Efficient AI Inference

Abstract The growing computational demands of AI raise a fundamental question: how much of the work performed during inference is actually necessary? Sparsity offers one answer by skipping zero-valued computations, yet many opportunities to avoid redundant work remain hidden in how models process information and how inference is executed. This talk explores these opportunities, beginning with fine-grained computational redundancy in the binary operations of spiking neural networks. It then moves beyond binary computation to examine structured redundancy in the richer, multi-bit representations of vision-language models, highlighting architectural approaches that translate these opportunities into lower inference latency. Finally, it considers how redundancy interacts with diffusion language model serving, taking an end-to-end system perspective that jointly accounts for throughput and latency under dynamic workloads. Together, these studies illustrate how recognizing and exploiting different forms of redundancy can guide the design of efficient AI architectures and systems beyond conventional zero-skipping. Speaker Bio Chiyue Wei is a Ph.D. candidate in Electrical and Computer Engineering at Duke University, advised by Professor Yiran Chen. His research focuses on computer architecture for efficient AI inference, with an emphasis on hardware–software co-design. He has ten papers published in or accepted to leading computer architecture conferences, including ISCA, MICRO, HPCA, and ASPLOS. His work received a Best Paper nomination at HPCA 2026. He has held internships at NVIDIA and Meta, working on AI inference software and accelerator architectures. He received his B.Eng. in Electronic Engineering from Tsinghua University in 2023.