A team from the Storage Research Group at Tsinghua University’s Department of Computer Science and Technology won first place at the ASPLOS 2025 / EuroSys 2025 contest on an optimized Neuron Kernel Interface (NKI) Implementation of Llama 3.2 1B (Inference). The competition was held during the 30th ACM International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS) and the 20th ACM European Conference on Computer Systems (EuroSys) from March 30 to April 3 in Rotterdam, Netherlands.

The winning team is composed of Ph.D. students Shiwei Gao, Ruwen Fan, and their colleagues, supervised by Prof. Jiwu Shu and Youyou Lu from the Department of Computer Science and Technology. The team distinguished themselves among competitors from renowned institutions, including Cornell University, New York University, the University of Wisconsin-Madison, King Abdullah University of Science and Technology, Yonsei University, Peking University, and Shanghai Jiao Tong University.

The contest challenged teams to develop the fastest inference implementation of the Llama 3.2 1B model using Amazon Web Services’ Neuron Kernel Interface (NKI) on a single Trainium1 (trn1) chip. Leveraging the AWS-provided NKI framework, the Tsinghua team implemented techniques such as computation tiling, instruction merging, and operator fusion, achieving a remarkable 1.1x performance improvement over Amazon’s baseline implementation.

ASPLOS and EuroSys, both hosted by the Association for Computing Machinery (ACM) with over two decades of history, are premier conferences in computer architecture and systems.

Editor: Li Han