Optimization of Large Models for Efficient Inference: Algorithm, Compiler, and System Co-Design - Gao, Shengyi (Journal of Computer, Signal, and System Research, 2025)
Large language models have achieved remarkable capabilities in natural language processing, yet their inference cost remains a significant challenge due to high computation, memory usage, and latency. This study presents a cross-layer co-optimization framework that integrates algorithmic, compiler, and system-level strategies to enhance inference efficiency. Algorithmic techniques, including structured pruning, sparsity, quantization, and dynamic inference, reduce computational workload and memory footprint. Compiler optimizations, such as operator fusion, graph rewriting, and layout specialization, translate algorithmic improvements into hardware-efficient execution. System-level strategies, encompassing parallel execution, memory management, and KV cache optimization, further improve resource utilization and reduce latency. The framework synergistically coordinates these layers, providing a theoretically grounded approach for reducing FLOPs, memory consumption, and inference latency. Its adaptability extends to cloud, edge, and interactive deployment scenarios, offering a unified methodology for efficient and scalable large-model inference. This work contributes a systematic and extensible pathway for accelerating model inference without relying on empirical performance measurements.